Module Development Timeline
Overview of the Materials Development Timeline
Each module development team will follow a series of steps and checkpoints designed to 1) align the materials development process with the overall GETSI project goals and timeline and 2) allow for timely interaction among the materials development teams, assessment team, UNAVCO, SERC staff, and technical advisors. After your proposal has been accepted and your contract is initiated, there are three major phases of developing, testing, and revising materials. After these phases are complete your, materials will be published on the GETSI website and will be publicly available for use.
Phase 1: Materials in Development (as a team)
Each team's authors work together to create detailed descriptions of materials for faculty use and student versions of the materials when necessary. The GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) outlines the expectations for how the materials will be created. It is based on the InTeGrate Materials Development and Refinement Rubric with minor project-specific differences. The GETSI Project Manager, Beth Pratt-Sitaula, and the team's assessment consultant provide guidance to the team to help them understand and meet the elements of the rubric. At the end of Phase 1 the materials will need to pass through the rubric, demonstrating that they have all of the desired attributes. In Phase 2, student data will be collected, so authors must prepare by obtaining IRB approval from their institution (more details below). Find GETSI-specific IRB documents at the top of the Forms and Documents page.Obtaining IRB Approval
Because the student assessment data is being used for research, we need to consider how human research guidelines apply. Most institutions have Institutional Review Boards that oversee this sort of data collection. Each institution implements this differently so you will need to explore the situation at your institution. Carleton College has approved the GETSI project as EXEMPT Carleton-UNAVCO IRB exempt letter (Acrobat (PDF) 62kB Feb25 14) (so have Indiana University and Mt San Antonio College). This approval may be sufficient for you to provide the collected student data to the InTeGrate project. However, each person collecting student data needs to check with their institution as the institution may require a separate IRB application.
Although Carleton College approved the application as EXEMPT, you still need to inform your students about this research so that the students have the option to not have their student assessments included. This can be completed in two ways and you will need to select the option that aligns with your institution's interpretation of the EXEMPT status:
The completed IRB application for Carleton College can be found on the Forms and Documents (opens in a new window) page and can be used as reference if you are required to complete the process on your own at your institution.
- Signed consent: In this case copies of the signed consent forms (with student names and GETSI ID's) would need to be sent and filed at the SERC office. Carleton College prefers signed student consent forms as the means to ensure that students are informed and have the opt-out option. Example Implied Consent Letter (general) (Microsoft Word 26kB Feb25 14)
- Implied consent: In this case, a list of students who have opted out (with their correlating GETSI ID's) should be provided to the SERC office. Example Signed Consent Form (MtSAC) (Microsoft Word 2007 (.docx) 17kB Apr19 15)
If your institution does not have an Institution Review Board then the Carleton IRB approval is sufficient, as this would fall under Carleton's multi-site involvement of assessment research. You will still need to provide students a consent form or letter so they have an opportunity to opt out (see above) and to share the Carleton Exempt letter as well as the student consent form you will be using with your campus leader.
Phase 2: Classroom Pilot (individually)
During the classroom pilot phase, module authors take their newly developed curriculum and try it out in their own classroom. Authors collect data that supports the evaluation of the team's materials and of the projects overall goals. Learn more about the design of the assessment and evaluation. Piloting allows authors to understand and document the learning that is taking place in the classroom. This helps ensure that the desired level of learning is being achieved and demonstrates pieces of the curriculum that may need revisions. Since each team member individually pilots the materials, there will be multiple sets of data to inform the material revisions. The classroom pilots also contribute to the evaluation of the overall GETSI project effort by gathering data that is common to all GETSI classroom pilots. Read more about the assessment data being collected and how it relates to your module and the overarching project on the Assessment Data page.
Phase 3: Post-Pilot Materials Revision (as a team)
The materials development team uses data collected during their classroom pilot along with assessment team feedback from the rubric review to guide the team toward meaningful revisions. Authors create individual case studies based on experiences and reflections from implementing the materials, and finalize the information that supports other faculty in using the materials. By the end of the process, the teaching materials and case studies will become publicly-available web pages that other faculty may access and implement in their own classes.
The show/hide links in each section outline the complete steps involved in passing the different phases of the materials development process and are separated by project roles. We encourage you to familiarize yourself with your own responsibilities as well as those of others.
To begin materials development, teams will need to do the following: confirm the topic of the module, provide a short description of the module that defines its scope and its central pedagogic features, describe how the materials the team develops address the guiding principles of GETSI/InTeGrate teaching materials, provide contact information for team members, and indicate the details about the course in which each team member will test the materials. This information will be used to establish the contract for each team member to ensure payment of the stipend. Each team member should understand the terms and conditions of the contract prior to the completion of this step.
Much, if not all, of this information will be completed at the face-to-face new author development meeting.
Materials Team Responsibilities:
- Input the information critical to initiating your contract. It should appear on your team reporting and checkpoint 1 pages, including:
- Summary of module (checkpoint 1 work page)
- How team's module addresses the guiding principles of GETSI/InTeGrate teaching materials (checkpoint 1 work page)
- Timeline for when the team estimates completing checkpoints 1-4 (reporting)
- Contact information for team members (reporting)
- Information about course in which each team member will be teaching GETSI material (reporting)
- Familiarize yourself with the Content Management System (CMS). You may be tempted to work in Word or Google Docs (or other software with which you are familiar), but this will hamper your progress and your ability to communicate with your assessment team consultant.
- Learn about Working Together as an InTeGrate Team.GETSI Project Manager Responsibilities (Beth Pratt-Sitaula): Welcome and orient everyone to the details of materials authoring. Review that all required information is available on the team reporting pages and in line with the project expectations:
- Summary of module (top of checkpoint 1 page)
- How the team's module addresses the guiding principles (top of reporting page)
- Timeline for when the team estimates completing checkpoints 1-4 (within each checkpoint area on reporting page)
- Contact information for team members (bottom reporting page)
- Course information in which each team member will be teaching GETSI material (bottom of reporting page)
If there are issues that need to be addressed prior to contract initiation, communicate with the team to assist them in making these changes. If the team's information is complete and passes your initial review, send an email via the team's email list that (a) signals the UNAVCO office that contracts can be sent to team members, and (b) signals the assessment person to familiarize themselves with the materials summary, timeline, and how the module addresses the guiding principles.Assessment Consultant Responsibilities: Once assigned to a materials development team, familiarize yourself with the materials summary, how the materials address the guiding principles, and timeline. If you have not already made initial contact with materials team, use the team's email list to do so now. Alert the GETSI leadership if you have any major concerns with the content thus far.Webteam/SERC Responsibilities: Set up the initial webspace and provide a tutorial to the CMS as needed.
Phase 1: Materials In Development
As the team develops materials, they will reach various checkpoints where feedback will be given from the overall GETSI Project Manager (Beth), the team's assessment consultant (Ellen or Stuart), and the team's webteam liaison (Molly). The purpose of the checkpoints is to ensure that the team has adequate input and support to develop successful materials as described by the GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) and is moving toward successfully passing rubric based review at Checkpoint 4. By checking the work against the rubric several times throughout development, there is a much smaller chance of having to make major revisions once the module is thought to be complete.
Checkpoint 1: Goals, Outline, Summative Assessment, and a Plan for Development
The purpose of this checkpoint is to review the initial design and scope of the materials, as well as the timeline and communication plan. This checkpoint insures that the team has a common vision and plan for creating the materials and that all support people understand and can meet the timeline.
The materials development team will have succinctly summarized the module content and how the module will address the guiding principles as part of the contract set up process. The next step in writing the module is for each team to develop module level goals, an overall outline of the materials, and a draft of their summative assessment (note: the summary and guiding principles were covered prior to this step, but may need to be modified as materials are further developed). This information should be recorded on the Checkpoint 1 Workpage in the team's module pages. This work will help the materials development team, GETSI Project Manager, assessment consultant, and webteam liaison all have a better understanding of the scope of the work being done and its alignment with the GETSI rubric.
At this point, the materials development team should also establish a timeline, ways to communicate with one another, and strategies for managing their collective progress. When the team's material is ready for review, the team should inform Beth and the assessment consultant via the team's email list (listed on the Materials in Progress page).
Timing for this checkpoint:
- This checkpoint should be completed as much as possible during the in-person development workshop.
- It will take your assessment consultant and Beth approximately 1-2 weeks to review your work.
Materials Team Responsibilities: By the end of Checkpoint 1, the following items should be recorded in your module webspace:
- a list of module level learning goals (checkpoint 1 workpage)
- a draft of summative assessment(s) for the module (checkpoint 1 workpage)
- a module outline (checkpoint 1 workpage)
- any edits to short description/summary and guiding principles (checkpoint 1 workpage)
- a team communication plan (bottom of team reporting page)
- a team geodesy data plan (bottom of team reporting page)
When your team feels you have reached Checkpoint 1 you will need to:
- Contact Beth via the team's email list. She will review the new content on your webspace. At this point, we are checking that the scope and organization of the materials match critical aspects of the GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) prior to further development and testing. Beth will either make suggestions for revisions or pass your module to your assessment consultant for review.
- Contact the GETSI webteam liaison and have a conversation about the structure of your module. This will allow her to create the necessary 'empty spaces' to hold your planned activity sheets (for instructions to the faculty) and student materials.
- Begin investigating the Institutional Review Board (IRB) approval process for your institution. This process is highly varied at each institution and can take a long time.
GETSI Project Manager Responsibilities: Review the scope of the material on the checkpoint 1 Workpage for the module goals, outline, and summative assessment, as well as the team communication plan (on team reporting page) and approve that the material has met the requirements of Checkpoint 1. Once approved, inform the assessment consultant (via the team email list) that the description, goals, and timeline are ready for their review.Assessment Consultant Responsibilities: Review how the team is addressing the guiding principles, module goals, and outline, for alignment with the rubric. Review the plausibility of the summative assessment. Provide constructive feedback to ensure the material is in alignment with the GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17).
Webteam Responsibilities: Continue to provide support for using the CMS and provide feedback on the design of the team's webspace and help with the creation of activity sheet pages that will be used to describe the activities within the module.
Checkpoint 2: Breadth, Depth, and Assessment
The purpose of this checkpoint is to provide comprehensive feedback on the structure of the materials, the pedagogic design of the materials and assessments, and the alignment of assessments, materials and goals. Checkpoint 2 is designed to occur at the earliest point in the materials development process, when it is possible to provide detailed and comprehensive feedback. The goal is to spot any developing problems early enough to prevent unnecessary work.
In order to be able to evaluate the team's progress towards meeting the design rubric, the GETSI Project Manager and assessment consultant will be looking for both breadth and depth in the module by checkpoint 2, including the assessments. Specifically, the team's web pages should include:
- a complete outline that shows the breadth of the module (overview page), including activity descriptions, pedagogic strategies, planned assessment strategies, and descriptions of any materials that will be developed for students (detailed in unit pages).
- at least one complete unit that includes faculty materials, student materials, and formative and summative assessments with rubrics to display the depth of your materials (work with the webteam liaison if you have questions, and refer to the Teaching Materials Format page)
- complete example(s) of a summative assessment(s) that will address the module-level goals (linked from the assessment page)
Once these pieces are complete, the team should communicate with the GETSI Project Manager (Beth) via the team's email list to initiate the review of the materials. She will review and provide feedback for revisions and pass them to the assessment consultant when the material is ready for review. The assessment consultant will score the materials using the GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) and provide constructive feedback.
Timing for this checkpoint:
- This checkpoint should occur about half-way through the materials development process, an absolute minimum of 3-4 weeks prior to checkpoint 3, 4-6 weeks prior to materials review, and 11-15 weeks prior to the earliest pilot test.
- It will take your assessment team member and team leader approximately 1-2 weeks to score your work.Materials Team Responsibilities: Once your module has achieved both breadth and depth (and everything is recorded on your webspace as listed above), communicate with the GETSI Project Manager that your materials are ready for Checkpoint 2 review via your team email list.
When you have reached checkpoint 2, initiate the Institutional Review Board (IRB) approval process.GETSI Project Manager Responsibilities: Review the completed materials for the module and either make suggestions for revisions, or pass the materials to the team's assessment consultant for review.Assessment Consultant Responsibilities: Once the team lead informs you that the materials are ready for review, score the module using the rubric. This will help you provide specific and constructive feedback with respect to the trajectory of their module development.
Webteam Responsibilities: Provide support for the CMS as needed.
Checkpoint 3: Materials Nearly Complete (75-90%)
The purpose of this checkpoint is to provide feedback on any final elements that are likely to prevent the materials from passing the rubric and finalize the timeline for the review process.
The materials development team continues working to create materials. At this checkpoint, some portions of the materials may be complete and others may be mostly complete, however, no individual pieces should be missing entirely. Your webspace should include full descriptions of activities, pedagogic approaches, and all assessments with needed rubrics.
Once materials are nearly complete, the team communicates with the team leader via the team's email list to initiate another review of the materials and to set the date when Checkpoint 4 (Materials Review) will occur.
Timing for this checkpoint:
- This checkpoint should be completed 1 to 2 weeks prior to the materials review (Checkpoint 4) deadline, and a absolute minimum of 7-9 weeks prior to the earliest classroom pilot date.
- It will take your assessment team member and team leader approximately 1-2 weeks to review your work.Materials Team Responsibilities:
- Alert the team lead via the team's email list that the module is ready for Checkpoint 3 review.
- Confirm the final review date when materials will be measured against the rubric.
- Provide a fully developed summative assessment (opens in a new window) on the assessment page in your module (if you have more than one, list them all). Provide details on how and when the summative assessment will be administered during your classroom pilot (e.g. after unit 3 as a graded exam question).
- If your materials are being reviewed close to the time of your classroom pilot (e.g. within 9 weeks), remember to address the data collection steps required to Collect Data in your Classroom (opens in a new window). Please use the Course Set up Form.
- Continue to work on gaining IRB approval if this is not yet complete.GETSI Project Manager Responsibilities: Review and provide feedback on all the available content (in addition to any comments and discussions that have already occurred) as well as module length (in terms of realistic teaching time). Review the module and either make suggestions for revisions, or pass the module to the team's assessment consultant for review (via an email to the team's email list).Assessment Consultant Responsibilities: Once the team lead signals the materials are ready for review, review available materials to ensure they are poised to pass the rubric. Alert the team of any issues or concerns. Verify the final review date with Ellen in order to line up all needed assessment reviewers.
- Continue to provide support for the CMS as needed.
- Verify that all summative assessments are clearly labeled/linked on the assessment page of the module.
- Prior to reaching Checkpoint 4, the webteam liaison also reads through the module and provides formatting and technical assistance as needed (checking links, adding pages to the navigation menu, editing page titles, removing stock text, ensuring that uploaded documents open, etc.)
Checkpoint 4: Materials Review and Revisions
This checkpoint is actually an interval of time over which the materials to be reviewed are finalized on the website, the review is conducted, and revisions necessary to pass the rubric are completed.
The materials development team enters the checkpoint when the materials for review are fully available in the module pages. Student worksheets are uploaded as needed, activity descriptions are completed, and the overview of the materials on the module overview page are sufficient for another instructor to understand how the parts of the module build upon one another to create a cohesive module. Once complete, the materials are reviewed by the GETSI Project Manager. The team may be asked to make revisions or additions. When the Project Manager has determined that the materials are ready for review, she initiates the review by contacting the assessment consultant. The materials are reviewed using the GETSI Materials Development Rubric (Microsoft Word 2007 (.docx) 134kB Dec7 17) by three members of the assessment team: the assessment consultant and two other InTeGrate assessment team members
Timing for this checkpoint:
- This checkpoint should be completed a minimum of 7 to 9 weeks prior to testing- which allows time for reviews and revisions. This is only enough time if there are no major issues with the module.
- It will take the team leader and assessment team at least 2 weeks to review your work (from the scheduled date).Materials Team Responsibilities:
- Ensure your team has responded to any assessment consultant feedback that was made during checkpoints 1-3; if you choose not to change materials based on feedback, make sure you clarify the reasoning for this decision and communicate this to the GETSI Project Manager.
- Be sure that formatting is similar among all the module pages and will support easy use and review of all of your materials. Ask the webteam liaison for assistance as needed.
- Email the GETSI Project Manager via the team's email list to let them know your module is ready for review.
- After review is complete, revise materials as needed. You may need to create a plan with your assessment consultant and GETSI Project Manager to complete revisions.
- Complete the Materials Development Reflection Survey (opens in a new window) after materials have passed the review and before teaching.
- Prior to the beginning of the term during which you plan to pilot your materials, follow the initial data collection steps (opens in a new window). Note that you may need to wait to complete this step until closer to the start of the term in which you will individually pilot your materials.
GETSI Project Manager Responsibilities: Review the module and either make suggestions for revisions, or pass the module to the team's assessment consultant for review (via team email list).
Assessment Team Responsibilities: You and 2 other assessment team members review the team's materials based on the rubric using the online review tool. Within 2 weeks of beginning review, provide the team with the review feedback and results.
Materials Team Receives first payment upon completing Checkpoint 4
Phase 2: Classroom Pilot
During Phase 2 you will teach with your materials and collect data that your team will use to improve the materials. These data are also used to evaluate the project as a whole. Learn more about how these data will help you and the project.
Pilot testing steps occur for each individual author (not as a team). Please note that there are data collection steps that need to be addressed prior to the start of the semester in which you plan to pilot the module in your class (see the "Data Check A: Before your course begins" section below in the Outline of Piloting Data Checks). Depending on timing, this may occur after Checkpoint 3 or 4. Your webteam liaison can help you determine when you should complete these steps. Read more about Collecting Data in your Classroom.
Data Check A: Before your course begins. Read more about Data Check A
- Obtain IRB approval (if not already complete)
- Decide how you will provide data to GETSI/InTeGrate
- Decide on a student ID method
- Decide on a per-student measure of assignment completion
- Complete the GETSI Course Setup Form.
Data Check B: At the beginning of your course (first week of class) Read more about Data Check B
- Collect student consent forms (if needed) and provide them to the SERC office
- Submit initial roster of student ID's
- Direct students to initial attitudinal survey
- Administer the initial GLE multiple choice questions
Data Check C: While you are teaching GETSI curriculum Read more about Data Check C
- Administer and collect summative assessments
Data Check D: At the end of your course Read more about Data Check D
- Administer the final GLE multiple choice questions
- Administer two common essay questions (as part of a high stakes assessment)
- Direct students to complete the final attitudinal survey
- Provide a final course roster, (proxy) measure of student engagement, and student assessment responses
- Complete the post-pilot reflection survey on your individual reporting forms
- All student assessment data is submitted and recorded in the author's course status page
You will receive the results of the assessment as soon after testing by both authors as feasible. The Materials Team, Assessment Consultant, and GETSI Project Manager will hold a virtual meeting to discuss results. These results will include
- a summary of the GLE pre/post questions, two common essay questions (if used) and attitudinal data for your module situated in the results to date for the project as a whole.
- an evaluation by assessment team of how well the summative assessment functioned for determining whether students met the module goals.
Phase 3: Post-Pilot Materials Revision and Review
During this phase of the materials development process, the team will revise the materials based on the information gathered during the pilot testing including the experiences of the team in teaching with the materials and the assessment results. Authors will complete the materials that support other faculty in using the materials, including the instructor testimonials which describe the adaptation of the materials for each author's class. At the end of phase 3 the materials will be published on the project website.
Checkpoint 5: Revision Plan
This work begins with the development of a revision plan by the team. This revision plan is based on all three classroom pilot tests, and is informed by the data collected, reflections, case studies, and an intimate knowledge of what worked/didn't work with the module. Comparing and contrasting challenges and successes with team members will help development teams create a strong coordinated revision plan. Each team will need to define changes as either:
Authors will work together to develop the plan, including a timeline and assignment of specific responsibilities.
- Global: changes to be applied to the module as a whole.
- Individual: changes that may help individual teaching scenarios.
Materials Team Responsibilities: Your global revisions should be recorded as your group revision plan on your team's reporting page (checkpoint 5). Individual changes should be recorded in the applicable instructor stories page(s). The revision plan should include a timeline and the assignment of specific responsibilities.GETSI Project Manager Responsibilities: Provide feedback from the third pilot tester (if applicable). Review the revision plan to determine if it will address the range of challenges observed with the materials and to ensure that there is a realistic timeline and assignment of specific responsibilities.
Assessment Consultant Responsibilities: Provide feedback from your review of classroom testing data. Authors should combine this with their own experiences in teaching the materials to develop a revision plan.
Checkpoint 6: Individual Instructor Stories and Coordinated Revisions Completed
The goal of this checkpoint is to revise the materials and prepare them for final review. As is the case with journal reviews, the authors will need to prepare a letter indicating how they addressed the concerns raised by the assessment team.
Authors revise materials in keeping with their plan. In addition, they attend to the final edits and formatting described below in the Materials Checklist. They obtain iterative feedback from their team lead as they work. The team leader oversees editing to ensure materials from different teams share a common voice and are fully completed.
- All necessary student materials are available on-line and consistent with the GETSI/InTeGrate student materials format.
- All necessary instructor materials are available on-line and consistent with the GETSI/InTeGrate teaching materials format.
- Writing style and tone are appropriate for the material. Teaching materials should follow best practices for web writing and present an accessible and compelling case for their use. Materials directed at students should have a voice and presentation appropriate for use in a broad range of classroom.
- Spelling and grammar are correct.
- Navigation within the web pages follows GETSI/InTeGrate conventions and is complete and consistent.
- All links are functional.
- Appropriate attribution is given and sources cited throughout. Copyright information
- All teacher-only materials are identified and communicated to the team webteam member so they can be placed in a protected space (only accessible to instructors).
- Provide team's webteam member with three examples of the kind of image the team would like to use on their front page.
- All downloadable materials:
- are provided formats appropriate for their intended use.
- appropriately cite their sources.
- Figures and images have captions including attribution and references as appropriate.
- All included materials (including photos), both online and downloadable, have accurate licensing information and can be legally redistributed by GETSI.
- Individual instructor stories are finished.
When the authors feel the materials are finished, they provide a letter to their team leader summarizing the changes they have made in response to the initial assessment team feedback. The letter should detail rationale for both changes made *and* areas where they didn't make the requested changes, similar to a letter in response to peer review of a scientific paper. Specifically the letter will address:
- the editing suggestions from the initial Materials Development Rubric
- assessment team feedback based on in-class testing data
- issues and challenges that were called out in the piloting instructors' post-instruction reflection
- Beth notifies the webteam member that checkpoint 6 has been reached. The webteam member moves the module into final format including:
- front page layout with image selected in coordination with SERC
- instructor stories
- check to make sure formatting, including navigation is correct
- student materials cloned
- science reviewer group given read access to instructor and student materials
- materials tagged with controlled vocabularies
- The webteam member also generates a technical review document which contains a list of items that need attention from the authors: broken links, major formatting problems, prov/reuse info that is incomplete or seems incorrect. They send this document to the team email list with a 2 week deadline and document the date that the technical review was completed and when the author response is expected on the reporting page. They notify Beth that step 1 has been completed.
- Beth arranges copyediting and documents an expected completion date on the reporting page.
- Beth arranges to have a science review conducted (if neither of the authors are content experts in the topic). Beth identifies reviewers and lets web member know their email address/es so they can be added to the security group. Once the module has been sent out for review Beth documents an expected review completion date on the reporting page. When science reviews are complete Beth sends a summary of suggestions to the author email list with a 2 week deadline and documents this on the reporting page.
- When the 'authors should respond by' deadlines for steps 2 and 5 are known Beth will document an expected checkpoint 7 deadline on the reporting page.
Note that the webteam work in step 1 may largely (or even entirely) be completed before checkpoint 6 is reached. Steps 2-5 will occur in parallel.
Checkpoint 7: Final Assessment and Content Review and Final Revisions
The GETSI Project Manager (Beth) will manage the final three reviews (science review, technical review, copyediting review) and the Assessment Lead (Ellen) will manage the final assessment. They will work with the module team to ensure appropriate changes are made.
- The authors review the final materials ready for publication.
- Materials go live (publicly available).
- Authors complete the finished revising reflection survey.
Materials published on GETSI site and become freely available for use
Second payment is made when materials are published.