Module Development Timeline

Development Timeline

Overview of the Materials Development Timeline

Each module development team will follow a series of steps and checkpoints designed to 1) align the materials development process with the overall GETSI project goals and timeline and 2) allow for timely interaction among the materials development teams, assessment team, UNAVCO, SERC staff, and technical advisors. After your proposal has been accepted and your contract is initiated, there are three major phases of developing, testing, and revising materials. After these phases are complete your, materials will be published on the GETSI website and will be publicly available for use.

Phase 1: Materials in Development (as a team) - for GETSI Phase 2 this is expected to be ~8-12 months from February 2018
Each team's authors work together to create detailed descriptions of materials for faculty use and student versions of the materials when necessary. The GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) outlines the expectations for how the materials will be created. It is based on the InTeGrate Materials Development and Refinement Rubric with minor project-specific differences. The GETSI Project Manager, Beth Pratt-Sitaula, and the team's assessment consultant provide guidance to the team to help them understand and meet the elements of the rubric. At the end of Phase 1 the materials will need to pass through the rubric, demonstrating that they have all of the desired attributes. In Phase 2, the materials will be tested in each author's classroom and student data will be collected. In the past, authors had to obtain Institutional Review Board (IRB) approval from their own institution to collect this data. However, beginning January 2018, new IRB guidelines allow multi-site data collection to be covered by Carleton's IRB exempt status (click below to read more about the guidelines). For those locations with an IRB, you should inform your IRB that you are participating in this study and that Carleton is acting as the IRB of record.

Phase 2: Classroom Pilot (individually) - for GETSI Phase 2 this should be finished by April 2019 (or sooner!)
During the classroom pilot phase, module authors take their newly developed curriculum and try it out in their own classroom. Authors collect data that supports the evaluation of the team's materials and of the projects overall goals. Learn more about the design of the assessment and evaluation. Piloting allows authors to understand and document the learning that is taking place in the classroom. This helps ensure that the desired level of learning is being achieved and demonstrates pieces of the curriculum that may need revisions. Since each team member individually pilots the materials, there will be multiple sets of data to inform the material revisions. The classroom pilots also contribute to the evaluation of the overall GETSI project effort by gathering data that is common to all GETSI classroom pilots. Read more about the assessment data being collected and how it relates to your module and the overarching project on the Assessment Data page.

Phase 3: Post-Pilot Materials Revision (as a team) - for GETSI Phase 2 this should be completed by August 1, 2019 (or sooner!)
The materials development team uses data collected during their classroom pilot along with assessment team feedback from the rubric review to guide the team toward meaningful revisions. Authors create individual case studies based on experiences and reflections from implementing the materials, and finalize the information that supports other faculty in using the materials. By the end of the process, the teaching materials and case studies will become publicly-available web pages that other faculty may access and implement in their own classes.

The show/hide links in each section outline the complete steps involved in passing the different phases of the materials development process and are separated by project roles. We encourage you to familiarize yourself with your own responsibilities as well as those of others.

Materials Team Initiates Work and Contract Initiated

As part of the application process, teams confirmed the topic of the module, provided a short description of the module that defines its scope and its central pedagogic features, described how the materials the team develops address the guiding principles of GETSI/InTeGrate teaching materials, provided contact information for team members, and indicated the details about the course in which each team member will test the materials. This information will be used to establish the contract for each team member to ensure payment of the stipend. Each team member should understand the terms and conditions of the contract prior to the completion of this step.

Much, if not all, of this information will be completed at the face-to-face new author development meeting.

Phase 1: Materials In Development

As the team develops materials, they will reach various checkpoints where feedback will be given from the overall GETSI Project Manager (Beth), the team's assessment consultant (Ellen or Stuart), and the GETSI webteam liaison (Monica). The purpose of the checkpoints is to ensure that the team has adequate input and support to develop successful materials as described by the GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) and is moving toward successfully passing rubric based review at Checkpoint 4. By checking the work against the rubric several times throughout development, there is a much smaller chance of having to make major revisions once the module is thought to be complete.

Checkpoint 1: Goals, Outline, Summative Assessment, and a Plan for Development

The purpose of this checkpoint is to review the initial design and scope of the materials, as well as the timeline and communication plan. This checkpoint insures that the team has a common vision and plan for creating the materials and that all support people understand and can meet the timeline.

The materials development team will have succinctly summarized the module content and how the module will address the guiding principles as part of the contract set up process. The next step in writing the module is for each team to develop module level goals, an overall outline of the materials, and a draft of their summative assessment (note: the summary and guiding principles were covered prior to this step, but may need to be modified as materials are further developed). This information should be recorded on the Checkpoint 1 Workpage in the team's module pages. This work will help the materials development team, GETSI Project Manager, assessment consultant, and webteam liaison all have a better understanding of the scope of the work being done and its alignment with the GETSI rubric.

At this point, the materials development team should also establish a timeline, ways to communicate with one another, and strategies for managing their collective progress. When the team's material is ready for review, the team should inform Beth and the assessment consultant via the team's email list (listed on the Materials in Progress page).

Timing for this checkpoint:

  • This checkpoint should be completed as much as possible during the in-person development workshop.
  • It will take your assessment consultant and Beth approximately 1-2 weeks to review your work.

Checkpoint 2: Breadth, Depth, and Assessment

The purpose of this checkpoint is to provide comprehensive feedback on the structure of the materials, the pedagogic design of the materials and assessments, and the alignment of assessments, materials and goals. Checkpoint 2 is designed to occur at the earliest point in the materials development process, when it is possible to provide detailed and comprehensive feedback. The goal is to spot any developing problems early enough to prevent unnecessary work.

In order to be able to evaluate the team's progress towards meeting the design rubric, the GETSI Project Manager and assessment consultant will be looking for both breadth and depth in the module by checkpoint 2, including the assessments. Specifically, the team's web pages should include:

  • a complete outline that shows the breadth of the module (overview page), including activity descriptions, pedagogic strategies, planned assessment strategies, and descriptions of any materials that will be developed for students (detailed in unit pages).
  • at least one complete unit that includes faculty materials, student materials, and formative and summative assessments with rubrics to display the depth of your materials (work with the webteam liaison if you have questions, and refer to the Teaching Materials Format page)
  • complete example(s) of a summative assessment(s) that will address the module-level goals (linked from the assessment page)

Once these pieces are complete, the team should communicate with the GETSI Project Manager (Beth) via the team's email list to initiate the review of the materials. She will review and provide feedback for revisions and pass them to the assessment consultant when the material is ready for review. The assessment consultant will score the materials using the GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) and provide constructive feedback.

Timing for this checkpoint:

  • This checkpoint should occur about half-way through the materials development process, an absolute minimum of 3-4 weeks prior to checkpoint 3, 4-6 weeks prior to materials review, and 11-15 weeks prior to the earliest pilot test.
  • It will take your assessment team member and team leader approximately 1-2 weeks to score your work.

Checkpoint 3: Materials Nearly Complete (75-90%)

The purpose of this checkpoint is to provide feedback on any final elements that are likely to prevent the materials from passing the rubric and finalize the timeline for the review process.

The materials development team continues working to create materials. At this checkpoint, some portions of the materials may be complete and others may be mostly complete, however, no individual pieces should be missing entirely. Your webspace should include full descriptions of activities, pedagogic approaches, and all assessments with needed rubrics.

Once materials are nearly complete, the team communicates with the team leader via the team's email list to initiate another review of the materials and to set the date when Checkpoint 4 (Materials Review) will occur.

Timing for this checkpoint:

  • This checkpoint should be completed 1 to 2 weeks prior to the materials review (Checkpoint 4) deadline, and a absolute minimum of 9-11 weeks prior to the earliest classroom pilot date.
  • It will take your assessment team member and team leader approximately 1-2 weeks to review your work.

Checkpoint 4: Materials Review and Revisions

This checkpoint is actually an interval of time over which the materials to be reviewed are finalized on the website, the review is conducted, and revisions necessary to pass the rubric are completed.

The materials development team enters the checkpoint when the materials for review are fully available in the module pages. Student worksheets are uploaded as needed, activity descriptions are completed, and the overview of the materials on the module overview page are sufficient for another instructor to understand how the parts of the module build upon one another to create a cohesive module. Once complete, the materials are reviewed by the GETSI Project Manager. The team may be asked to make revisions or additions. When the Project Manager has determined that the materials are ready for review, she initiates the review by contacting the assessment consultant. The materials are reviewed using the GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) by three members of the assessment team: the assessment consultant and two other InTeGrate assessment team members

Timing for this checkpoint:

  • This checkpoint should be completed a minimum of 7 to 9 weeks prior to testing- which allows time for reviews and revisions. This is only enough time if there are no major issues with the module.
  • It will take the team leader and assessment team at least 2 weeks to review your work (from the scheduled date).

Materials Team Receives first payment upon completing Checkpoint 4

Phase 2: Classroom Pilot

During Phase 2 you will teach with your materials and collect data that your team will use to improve the materials. These data are also used to evaluate the project as a whole. Learn more about how these data will help you and the project.

Pilot testing steps occur for each individual author (not as a team). Please note that there are data collection steps that need to be addressed prior to the start of the semester in which you plan to pilot the module in your class (see the "Data Check A: Before your course begins" section below in the Outline of Piloting Data Checks). Depending on timing, this may occur after Checkpoint 3 or 4. Your webteam liaison can help you determine when you should complete these steps. Read more about Collecting Data in your Classroom.

You will receive the results of the assessment as soon after testing by both authors as feasible. The Materials Team, Assessment Consultant, and GETSI Project Manager will hold a virtual meeting to discuss results. These results will include

  • a summary of the GLE pre/post questions, two common essay questions (if used) and attitudinal data for your module situated in the results to date for the project as a whole.
  • an evaluation by assessment team of how well the summative assessment functioned for determining whether students met the module goals.

Phase 3: Post-Pilot Materials Revision and Review

During this phase of the materials development process, the team will revise the materials based on the information gathered during the pilot testing including the experiences of the team in teaching with the materials and the assessment results. Authors will complete the materials that support other faculty in using the materials, including the instructor testimonials which describe the adaptation of the materials for each author's class. Please be sure you have signed the faculty consent form for these public self-authored stories. At the end of phase 3 the materials will be published on the project website.

Checkpoint 5: Revision Plan

This work begins with the development of a revision plan by the team. This revision plan is based on all three classroom pilot tests, and is informed by the data collected, reflections, case studies, and an intimate knowledge of what worked/didn't work with the module. Comparing and contrasting challenges and successes with team members will help development teams create a strong coordinated revision plan. Each team will need to define changes as either:

  1. Global: changes to be applied to the module as a whole.
  2. Individual: changes that may help individual teaching scenarios.
Authors will work together to develop the plan, including a timeline and assignment of specific responsibilities.

Checkpoint 6: Individual Instructor Stories and Coordinated Revisions Completed

The goal of this checkpoint is to revise the materials and prepare them for final review. As is the case with journal reviews, the authors will need to prepare a letter indicating how they addressed the concerns raised by the assessment team.

Authors revise materials in keeping with their plan. In addition, they attend to the final edits and formatting described below in the Materials Checklist. They obtain iterative feedback from their team lead as they work. The team leader oversees editing to ensure materials from different teams share a common voice and are fully completed.

Materials Checklist

  • All necessary student materials are available on-line and consistent with the GETSI/InTeGrate student materials format.
  • All necessary instructor materials are available on-line and consistent with the GETSI/InTeGrate teaching materials format.
  • Writing style and tone are appropriate for the material. Teaching materials should follow best practices for web writing and present an accessible and compelling case for their use. Materials directed at students should have a voice and presentation appropriate for use in a broad range of classroom.
  • Spelling and grammar are correct.
  • Navigation within the web pages follows GETSI/InTeGrate conventions and is complete and consistent.
  • All links are functional.
  • Appropriate attribution is given and sources cited throughout. Copyright information
  • All teacher-only materials are identified and communicated to the team webteam member so they can be placed in a protected space (only accessible to instructors).
  • Provide team's webteam member with three examples of the kind of image the team would like to use on their front page.
  • All downloadable materials:
    • are provided formats appropriate for their intended use.
    • appropriately cite their sources.
  • Figures and images have captions including attribution and references as appropriate.
  • All included materials (including photos), both online and downloadable, have accurate licensing information and can be legally redistributed by GETSI.
  • Individual instructor stories are finished and faculty consent signed (read more about IRB and consent). Submit your instructor stories with this form. If you want to see the components ahead of time, here is a Word version of the Instructor Story File (Microsoft Word 2007 (.docx) 113kB Nov1 16).

When the authors feel the materials are finished, they provide a letter to their team leader summarizing the changes they have made in response to the initial assessment team feedback. The letter should detail rationale for both changes made *and* areas where they didn't make the requested changes, similar to a letter in response to peer review of a scientific paper. Specifically the letter will address:

  • the editing suggestions from the initial Materials Development Rubric
  • assessment team feedback based on in-class testing data
  • issues and challenges that were called out in the piloting instructors' post-instruction reflection
The author team notifies Beth when this letter is ready. Beth initiates the 3 parallel elements of the final review: science review, technical review, and copy editing. These results in feedback to the team which they address to reach checkpoint 7.

Checkpoint 7: Final Assessment and Content Review and Final Revisions

The GETSI Project Manager (Beth) will manage the final three reviews (science review, technical review, copyediting review) and the Assessment Lead (Ellen) will manage the final assessment. They will work with the module team to ensure appropriate changes are made.

Materials published on GETSI site and become freely available for use

Second payment is made when materials are published.