Your Module/Course Development Timeline

×

Overview of the Materials Development Timeline

Each module/course development team will follow a series of steps and checkpoints designed to 1) align the materials development process with the overall InTeGrate project goals and timeline and 2) allow for timely interaction among the materials development teams, assessment team, InTeGrate leadership, and SERC staff. After your proposal has been accepted and your contract is initiated, there are three major phases of developing, testing, and revising materials. After these phases are complete your, materials will be published on the InTeGrate website and will be publicly available for use.

Phase 1: Materials in Development (as a team)
Each team's authors work together to create detailed descriptions of materials for faculty use and student versions of the materials when necessary. The Materials Development and Refinement Rubric describes the types of InTeGrate materials the project is creating. The team leader (Tim Bralower, Anne Egger, David Gosselin, David McConnell, John Taber) and the team's assessment consultant provide guidance to the team to help them understand and meet the elements of the rubric. At the end of Phase 1 the materials will need to pass through the rubric, demonstrating that they have all of the desired attributes. In Phase 2, student data will be collected, so authors must prepare by obtaining IRB approval from their institution.

Phase 2: Classroom Pilot (individually)
During the classroom pilot phase, module/course authors take their newly developed curriculum and try it out in their own classroom. Authors collect data that supports the evaluation of the team's materials and of the projects overall goals. Learn more about the design of the assessment and evaluation. Piloting allows authors to understand and document the learning that is taking place in the classroom. This helps ensure that the desired level of learning is being achieved and demonstrates pieces of the curriculum that may need revisions. Since each team member individually pilots the materials, there will be multiple sets of data to inform the material revisions. The classroom pilots also contribute to the evaluation of the larger InTeGrate project effort by gathering data that is common to all InTeGrate classroom pilots. Read more about the assessment data being collected and how it relates to your module/course and the overarching project on the Assessment Data page.

Phase 3: Post-Pilot Materials Revision (as a team)
The materials development team uses data collected during their classroom pilot along with assessment team feedback from the rubric review to guide the team toward meaningful revisions. Authors create individual case studies based on experiences and reflections from implementing the materials, and finalize the information that supports other faculty in using the materials. By the end of the process, the teaching materials and case studies will become publicly-available web pages that other faculty may access and implement in their own classes.

The show/hide links in each section outline the complete steps involved in passing the different phases of the materials development process and are separated by project roles. We encourage you to familiarize yourself with your own responsibilities as well as those of others.

Materials Team Application Accepted and Contract Initiated

To begin materials development, teams will need to do the following: confirm the title of the course/module, provide a short description of the module/course that defines its scope and its central pedagogic features, describe how the materials the team develops address the guiding principles of InTeGrate teaching materials, provide contact information for team members, and indicate the details about the course in which each team member will test the materials. This information will be used to establish the contract for each team member to ensure payment of the stipend. Each team member should understand the terms and conditions of the contract prior to the completion of this step.

Much, if not all, of this information will be collected at the face-to-face new author orientation meeting. If the team does not have a representative present at the new authors face-to-face meeting, you can submit the information using the materials set-up form.

Once this information is received (either at the meeting or via the form), the webteam liaison will build your team's initial webspace and email list, your contract will be drafted, and Stuart Birnbaum will assign the team an assessment consultant. While the team is waiting for this to take place, they can read the Information for New Authors and supporting pages to familiarize themselves with the authoring process.

Show Materials Team Responsibilities

Hide

Materials team responsibilities:

• Review the Materials Development Expectations and Payment web page (opens in a new window).
• As a team, discuss and complete the Module/Course Description and Timeline form (please only complete this form if you did not attend the new authors face-to-face orientation meeting where this information is collected/developed). The information critical to initiate your contract should appear on your team reporting page, including:
• Summary of course/module
• How team's course/module addresses the guiding principles of InTeGrate teaching materials
• Timeline for when the team estimates completing checkpoints 1-4
• Contact information for team members
• Course information in which each team member will be teaching InTeGrate material
• Create your SERC account (if you haven't already done so) and familiarize yourself with the Content Management System (CMS)
• Learn about Working Together as an InTeGrate Team.
• Schedule a tutorial with your webteam liaison to provide an orientation to the CMS and your module/course webspace. You may be tempted to work in Word or Google Docs (or other software with which you are familiar), but this will hamper your progress and your ability to communicate with your assessment team consultant. Working Together as an InTeGrate Team will help you get started working with your team on the CMS.

Show Team Leader Responsibilities

Hide

Team Leader Responsibilities (e.g. David McConnell, Anne Egger, David Gosselin, Tim Bralower, John Taber): Welcome and orient your team(s) to the details of materials authoring. Review that all required information is available on the team reporting pages and in line with the project expectations:

• Summary of course/module (top of checkpoint 1 page)
• How the team's course/module addresses the guiding principles (top of reporting page)
• Timeline for when the team estimates completing checkpoints 1-4 (within each checkpoint area on reporting page)
• Contact information for team members (team reporting page)
• Course information in which each team member will be teaching InTeGrate material (team reporting page)

If there are issues that need to be addressed prior to contract initiation, communicate with the team to assist them in making these changes. If the team's information is complete and passes your initial review, send an email via the team's email list that (a) signals the SERC office that contracts can be sent to team members, and (b) signals the assessment person to familiarize themselves with the materials summary, timeline, and how the course/module addresses the guiding principles.

Show Assessment Team Responsibilities

Hide

Assessment team responsibilities: Once assigned to a materials development team, familiarize yourself with the materials summary, how the materials address the guiding principles, and timeline. If you haven't already made initial contact with materials team, use the team's email list to do so now. Alert the team lead if you have any major concerns with the content thus far.

Show Webteam/SERC Responsibilities

Hide

Webteam/SERC responsibilities: Set up the initial webspace and provide a tutorial to the CMS if requested by materials team. When the team lead says the module/course is ready for a contract (via the team's email list), the SERC office will draft and send team members their contract.

Phase 1: Materials In Development

As the team develops materials, they will reach various checkpoints where feedback will be given from the overall team leader (e.g. David McConnell, Anne Egger, David Gosselin, Tim Bralower, John Taber), the team's assessment team consultant, and the team's webteam liaison The purpose of the checkpoints is to ensure that the team has adequate input and support to develop successful materials as described by the InTeGrate Materials Development and Refinement Rubric and is moving toward successfully passing rubric based review at Checkpoint 4. By checking the work against the rubric several times throughout development, there is a much smaller chance of having to make major revisions once the module/course is complete.

Checkpoint 1: Goals, Outline, Summative Assessment, and a Plan for Development

The purpose of this checkpoint is to review the initial design and scope of the materials, as well as the timeline and communication plan. This checkpoint insures that the team has a common vision and plan for creating the materials and that all support people understand and can meet the timeline.

The materials development team will have succinctly summarized the module/course content and how the module/course will address the guiding principles as part of the contract set up process. The next step in writing the module or course is for each team to develop module/course level goals, an overall outline of the materials, and a draft of their summative assessment (note: the summary and guiding principles were covered prior to this step, but may need to be modified as materials are further developed). This information should be recorded on the Checkpoint 1 Workpage in the team's course/module pages (note: teams led by McConnell and Egger that initiated work in May 2013 will complete this information within their module/course pages rather than in the Checkpoint 1 Workpage) . This work will help the materials team, team leader, assessment team consultant, and webteam liaison all have a better understanding of the scope of the work being done and its alignment with the rubric.

At this point, the materials development team should also establish a timeline, ways to communicate with one another, and strategies for managing their collective progress. When the team's material is ready for review, the team should inform the team leader via the team's email list (listed on the Materials in Progress page).

Timing for this checkpoint:

• This checkpoint should be completed after your contract is in place.
• It will take your assessment team member and team leader approximately 1-2 weeks to review your work.

Show Materials Team Responsibilities

Hide

Materials team responsibilities: By the end of Checkpoint 1, the following items should be recorded in your module/course webspace:

• a list of course/module level learning goals (checkpoint 1 workpage)
• a draft of summative assessment(s) for the course/module (checkpoint 1 workpage)
• a course/module outline (checkpoint 1 workpage)
• any edits to short description/summary (front page of the module)
  
• any edits to guiding principles (top of team reporting page)
• author contact info and piloting info (team reporting page)
  
• a team communication plan (team reporting page)

When your team feels you have reached Checkpoint 1 you will need to:

• Contact your team lead via the team's email list. They'll review the new content on your webspace. At this point, we are checking that the scope and organization of the materials match critical aspects of the InTeGrate Curriculum Development and Refinement Rubric prior to further development and testing. Your team lead will either make suggestions for revisions or pass your module/course to your assessment consultant for review.
• Contact your webteam support person and have a conversation about the structure of your course/module. This will allow them to create the necessary 'empty spaces' to hold your planned activity sheets (for instructions to the faculty) and student materials.
• Begin investigating the Institutional Review Board (IRB) approval process for your institution. This process is highly varied at each institution and can take a long time.

Show Team Leader Responsibilities

Hide

Team leader responsibilities: Review the scope of the material on the checkpoint 1 Workpage for the module/course goals, outline, and summative assessment, as well as the team communication plan (on team reporting page) and approve that the material has met the requirements of Checkpoint 1. Once approved, inform the assessment consultant (via the team email list) that the description, goals, and timeline are ready for their review.

Show Assessment Consultant Responsibilities

Hide

Assessment consultant responsibilities: Review how the team is addressing the guiding principles, module/course goals, and outline, for alignment with the rubric. Review the plausibility of the summative assessment. Provide constructive feedback to ensure the material is in alignment with the InTeGrate Materials Development and Design Rubric.

Show Webteam Responsibilities

Hide

Webteam responsibilities: Continue to provide support for using the CMS and provide feedback on the design of the team's webspace and help with the creation of activity sheet pages that will be used to describe the activities within the module/course.

Checkpoint 2: Breadth, Depth, and Assessment

The purpose of this checkpoint is to provide comprehensive feedback on the structure of the materials, the pedagogic design of the materials and assessments, and the alignment of assessments, materials and goals. Checkpoint 2 is designed to occur at the earliest point in the materials development process, when it is possible to provide detailed and comprehensive feedback. The goal is to spot any developing problems early enough to prevent unnecessary work.

In order to be able to evaluate the team's progress towards meeting the design rubric, the team leader and assessment consultant will be looking for both breadth and depth in the module or course by checkpoint 2, including the assessments. Specifically, the team's web pages should include:

• a complete outline that shows the breadth of the course or module (overview page), including activity descriptions, pedagogic strategies, planned assessment strategies, and descriptions of any materials that will be developed for students (detailed in unit/module pages).
• at least one complete activity (for a module) or module (for a course) that includes faculty materials, student materials, and formative and summative assessments with rubrics to display the depth of your materials (upload these in the course materials- work with your webteam consultant if you have questions, and refer to the Teaching Materials Format page)
• complete example(s) of a summative assessment(s) that will address the module- or course-level goals (linked from the assessment page)

Once these pieces are complete, the team should communicate with the team leader via the team's email list to initiate the review of the materials. The team leader will review and provide feedback for revisions and pass them to the assessment consultant when the material is ready for review. The assessment consultant will score the materials using the rubric and provide constructive feedback.

Timing for this checkpoint:

• This checkpoint should occur about half-way through the materials development process, an absolute minimum of 3-4 weeks prior to checkpoint 3, 4-6 weeks prior to materials review, and 11-15 weeks prior to the earliest pilot test.
• It will take your assessment team member and team leader approximately 1-2 weeks to score your work.

Show Materials Team Responsibilities

Hide

Materials team responsibilities: Once your course or module has achieved both breadth and depth (and everything is recorded on your webspace as listed above), communicate with your team lead that your materials are ready for Checkpoint 2 review via your team email list.

When you've reached checkpoint 2, initiate the Institutional Review Board (IRB) approval process (opens in a new window).

Show Team Leader Responsibilities

Hide

Team leader responsibilities: Review the completed materials for the module/course and either make suggestions for revisions, or pass the materials to the team's assessment consultant for review.

Show Assessment Consultant Responsibilities

Hide

Assessment consultant responsibilities: Once the team lead informs you that the materials are ready for review, score the course/module using the rubric. This will help you provide specific and constructive feedback with respect to the trajectory of their course/module development.

Show Webteam Responsibilities

Hide

Webteam responsibilities: Provide support for the CMS as needed.

Checkpoint 3: Materials Nearly Complete (75-90%)

The purpose of this checkpoint is to provide feedback on any final elements that are likely to prevent the materials from passing the rubric and finalize the timeline for the review process.

The materials development team continues working to create materials. At this checkpoint, some portions of the materials may be complete and others may be mostly complete, however, no individual pieces should be missing entirely. Your webspace should include full descriptions of activities, pedagogic approaches, and all assessments with needed rubrics.

Once materials are nearly complete, the team communicates with the team leader via the team's email list to initiate another review of the materials and to set the date when Checkpoint 4 (Materials Review) will occur.

Timing for this checkpoint:

• This checkpoint should be completed 1 to 2 weeks prior to the materials review (Checkpoint 4) deadline, and a absolute minimum of 7-9 weeks prior to the earliest classroom pilot date.
• It will take your assessment team member and team leader approximately 1-2 weeks to review your work.

Show Materials Team Responsibilities

Hide

Materials team responsibilities:

• Alert the team lead via the team's email list that the module/course is ready for Checkpoint 3 review.
• Confirm the final review date when materials will be measured against the rubric.
• Provide a fully developed summative assessment (opens in a new window) on the assessment page in your module/course (if you have more than one, list them all). Provide details on how and when the summative assessment will be administered during your classroom pilot (e.g. after unit 3 as a graded exam question).
• If your materials are being reviewed close to the time of your classroom pilot (e.g. within 9 weeks), remember to address the data collection steps required to Collect Data in your Classroom (opens in a new window).
• Continue to work on gaining IRB approval if this is not yet complete.

Show Team Leader Responsibilities

Hide

Team leader responsibilities: Review and provide feedback on all the available content (in addition to any comments and discussions that have already occurred) as well as module/course length (in terms of realistic teaching time). Review the module/course and either make suggestions for revisions, or pass the module/course to the team's assessment consultant for review (via an email to the team's email list).

Show Assessment Consultant Responsibilities

Hide

Assessment team responsibilities: Once the team lead signals the materials are ready for review, review available materials to ensure they are poised to pass the rubric. Alert the team of any issues or concerns. Verify the final review date with Stuart Birnbaum in order to line up all needed assessment reviewers.

Show Webteam Responsibilities

Hide

Webteam responsibilities:

• Continue to provide support for the CMS as needed.
• Verify that all summative assessments are clearly labeled/linked on the assessment page of the module/course.
• Prior to reaching Checkpoint 4, the webteam liaison also reads through the module/course and provides formatting and technical assistance as needed (checking links, adding pages to the navigation menu, editing page titles, removing stock text, ensuring that uploaded documents open, etc.)

Checkpoint 4: Materials Review and Revisions

This checkpoint is actually an interval of time over which the materials to be reviewed are finalized on the website, the review is conducted, and revisions necessary to pass the rubric are completed.

The materials development team enters the checkpoint when the materials for review are fully available in the module/course pages. Student worksheets are uploaded as needed, activity descriptions are completed, and the overview of the materials on the module/course overview page are sufficient for another instructor to understand how the parts of the module or course build upon one another to create a cohesive module or course. Once complete, the materials are reviewed by the team lead. The team may be asked to make revisions or additions by the team lead. When the team lead has determined that the materials are ready for review, he/she initates the review by contacting the assessment team. The materials are reviewed using the InTeGrate Curriculum Development and Refinement Rubric by three members of the assessment team: the assessment consultant and two other assessment team members

Timing for this checkpoint:

• This checkpoint should be completed a minimum of 7 to 9 weeks prior to testing- which allows time for reviews and revisions. This is only enough time if there are no major issues with the module/course.
• It will take the team leader and assessment team at least 2 weeks to review your work (from the scheduled date).

Show Materials Team Responsibilities

Hide

Materials team responsibilities:

• Ensure your team has responded to any assessment consultant feedback that was made during checkpoints 1-3; if you choose not to change materials based on feedback, make sure you clarify the reasoning for this decision and communicate this to your team lead.
• Be sure that formatting is similar among all the module or course pages and will support easy use and review of all of your materials. Ask your webteam liaison for assistance as needed.
• Email your team leader via the team's email list to let them know your module/course is ready for review.
• After review is complete, revise materials as needed. You may need to create a plan with your assessment and team leaders to complete revisions.
• Complete the Materials Development Reflection Survey after materials have passed the review and before teaching.
• Prior to the beginning of the term during which you plan to pilot your materials, follow the initial data collection steps. Note that you may need to wait to complete this step until closer to the start of the term in which you will individually pilot your materials.

Show Team Leader Responsibilities

Hide

Team leader responsibilities: Review the module/course and either make suggestions for revisions, or pass the module/course to the team's assessment consultant for review (via team email list).

Show Assessment Team Responsibilities

Hide

Assessment team responsibilities: You and 2 other assessment team members review the team's materials based on the rubric using the online review tool. Within 2 weeks of beginning review, provide the team with the review feedback and results.

Materials Team Receives first payment upon completing Checkpoint 4

Phase 2: Classroom Pilot

During Phase 2 you will teach with your materials and collect data that your team will use to improve the materials. These data are also used to evaluate the project as a whole. Learn more about how these data will help you and the project.

Pilot testing steps occur for each individual author (not as a team). Please note that there are data collection steps that need to be addressed prior to the start of the semester in which you plan to pilot the module/course in your class (see the "Data Check A: Before your course begins" section below in the Outline of Piloting Data Checks). Depending on timing, this may occur after Checkpoint 3 or 4. Your webteam liaison can help you determine when you should complete these steps. Read more about Collecting Data in your Classroom.

Show Outline of Piloting Data Checks A - D

Hide

You will receive the results of the assessment prior to the second face-to-face meeting of your materials development team. These results will include

• a summary of the GLE pre/post questions, two common essay questions and attitudinal data for your module or course situated in the results to date for the project as a whole.
• an independent scoring of the student responses to the summative assessments by multiple members of the assessment team.

Phase 3: Post-Pilot Materials Revision and Review

During this phase of the materials development process, the team will revise the materials based on the information gathered during the pilot testing including the experiences of the team in teaching with the materials and the assessment results. Authors will complete the materials that support other faculty in using the materials, including the instructor testimonials which describe the adaptation of the materials for each author's class. At the end of phase 3 the materials will be published on the project website.

Checkpoint 5: Revision Plan

This work begins with the development of a revision plan by the team, typically, at the second face-to-face meeting of your team. This revision plan is based on all three classroom pilot tests, and is informed by the data collected, reflections, case studies, and an intimate knowledge of what worked/didn't work with the module or course. Comparing and contrasting challenges and successes with team members will help development teams create a strong coordinated revision plan. Each team will need to define changes as either:

1. Global: changes to be applied to the module/course as a whole.
   
2. Individual: changes that may help individual teaching scenarios.

Authors will work together to develop the plan, including a timeline and assignment of specific responsibilities.

Show Materials Team Responsibilities

Hide

Materials team responsibilities: Your global revisions should be recorded as your group revision plan on your team's reporting page (checkpoint 5). You're team will also need to answer the reflection questions found in that section. Individual changes should be recorded in the applicable case study page(s). The revision plan should include a timeline and the assignment of specific responsibilities.

Show Team Lead Responsibilities

Hide

Team Lead responsibilities: Provide feedback from the observation team (if applicable). Review the revision plan to determine if it will address the range of challenges observed with the materials and to ensure that there is a realistic timeline and assignment of specific responsibilities.

Show Assessment Consultant Responsibilities

Hide

Assessment Consultant responsibilities: Provide feedback from your review of classroom testing data. Authors should combine this with their own experiences in teaching the materials to develop a revision plan.

Checkpoint 6: Individual Instructor Stories and Coordinated Revisions Completed

The goal of this checkpoint is to revise the materials and prepare them for final review. As is the case with journal reviews, the authors will need to prepare a letter indicating how they addressed the concerns raised by the assessment team.

Authors revise materials in keeping with their plan. In addition, they attend to the final edits and formatting described below in the Materials Checklist. They obtain iterative feedback from their team lead as they work. The team leader oversees editing to ensure materials from different teams share a common voice and are fully completed.

Materials Checklist

• All necessary student materials are available on-line and consistent with the InTeGrate student materials format.
• All necessary instructor materials are available on-line and consistent with the InTeGrate teaching materials format.
• Writing style and tone are appropriate for the material. Teaching materials should follow best practices for web writing and present an accessible and compelling case for their use. Materials directed at students should have a voice and presentation appropriate for use in a broad range of classroom.
• Spelling and grammar are correct.
• Navigation within the web pages follows InTeGrate conventions and is complete and consistent.
• All links are functional.
• Appropriate attribution is given and sources cited throughout Copyright information.
• All teacher-only materials are identified and communicated to the team webteam member so they can be placed in a protected space (only accessible to instructors).
• Provide team's webteam member with three examples of the kind of image the team would like to use on their front page.
  
• All downloadable materials:
• are provided formats appropriate for their intended use.
• appropriately cite their sources.
• Figures and images have captions including attribution and references as appropriate.
• All included materials (including photos), both online and downloadable, have accurate licensing information and can be legally redistributed by InTeGrate.
• Individual instructor studies are finished.

When the authors feel the materials are finished, they provide a letter to their team leader summarizing the changes they've made in response to the initial assessment team feedback. The letter should detail rationale for both changes made *and* areas where they didn't make the requested changes, similar to a letter in response to peer review of a scientific paper. Specifically the letter will address:

• the editing suggestions from the initial Materials Development Rubric
• assessment team feedback based on in-class testing data
• issues and challenges that were called out in the piloting instructors' post-instruction reflection

The Team Leader notifies Krista Herbstrith when this letter is ready. Krista initiates the 3 parallel elements of the final review: technical review, science review and copyediting. These results in feedback to the team which they address to reach checkpoint 7.

Show detailed process between checkpoint 6 and 7

Hide

Checkpoint 7: Final Assessment and Content Review and Final Revisions

If necessary, feedback from the three reviews (science review, technical review, copyediting review) is sent to the team leader, who works with the module/course team to ensure appropriate changes are made. Stuart Birnbaum apprises Cathy Manduca of final completion of the instructional materials. The team leader apprises Cathy Manduca of final completion of the instructor materials.

• The authors review the final materials ready for publication.
• Materials go live (publicly available).
• Authors complete the finished revising reflection survey.

Materials published on InTeGrate site and become freely available for use

Second payment is made when materials are published.