TIDeS Rubric for Curriculum Development

Introduction

Rubrics are well-established as useful tools to communicate and assess learning goals (Andrade, 2005; Brookhart and Chen, 2015), and have also been used to guide curriculum design and identify needs for professional development (Steer et al., 2019). We want the instructional materials we develop in this project to center on investigation and design, to incorporate discourse, to promote an inclusive learning environment, and to make use of research-based practices in teaching (e.g. learning goals, high-quality resources, aligned assessments). In order to assure that the materials that are developed meet these goals, we developed a rubric against which the materials can be assessed. To help materials developers meet the standards of the rubric, we also developed guidelines that explain each criterion and give examples of how to meet it.

Developing and testing the TIDeS rubric

We began with the InTeGrate Materials Development and Refinement Rubric (Microsoft Word 2007 (.docx) 113kB Jan18 14), which consists of six sections:

• Guiding principles (specific to the InTeGrate project)
• Learning objectives and goals
• Assessment and measurement
• Resources and materials
• Instructional strategies
• Alignment

The InTeGrate rubric had been used successfully in the creation of InTeGrate teaching materials to ensure that they met the project goals (Steer et al., 2019). However, we knew we would need to modify it because the project goals differed.

Developing the guiding principles: The project leadership worked to develop the guiding principles, considering that they would guide all aspects of the TIDeS project, not just the materials development. We developed four guiding principles that align with recommendations from the 2018 National Academies report Science and Engineering for Grades 6-12: Investigation and Design at the Center and the Principles and Strategies for Inclusive Teaching from the University of Michigan Center for Research on Learning and Teaching.

Modifying other rubric sections: In every other section of the rubric, we revised, added, and removed criteria in order to (1) better emphasize the guiding principles in all sections, (2) add new criteria that were not part of the InTeGrate rubric, and (3) clarify criteria that InTeGrate authors found confusing and/or challenging.

To assess content validity, the draft rubric was presented to the TIDeS advisory board and their feedback was incorporated into a revised version.

Once the Materials Development criteria were developed, the evaluation team collaborated with project leadership to develop guidelines, collect examples, and assess inter-rater reliability. Five team members (including project leadership, the Science Education Specialists (SES), and evaluation team members) independently applied the rubric to assess existing modules developed as part of the InTeGrate project. The value of Cronbach's alpha across the 25 criteria was α = 0.88, indicating strong inter-rater reliability. As part of this process, we also developed and refined explanations of each criterion and linked to exemplars from several different projects.

Final rubric

You can download the TIDeS rubric guidelines (Acrobat (PDF) 318kB Sep11 23).

Implementing the rubric

The rubric is being used by the TIDeS materials developers to guide their work, and to guide the leadership in offering professional development that meets the needs of the developers. When teams feel their materials are complete, they will be assessed by an external assessment consultant against the rubric.

References

Andrade, H. G., 2005, Teaching With Rubrics: The Good, the Bad, and the Ugly: College Teaching, v. 53, no. 1, p. 27-31. doi: 10.3200/CTCH.53.1.27-31

Brookhart, S. M., and Chen, F., 2015, The quality and effectiveness of descriptive rubrics: Educational Review, v. 67, no. 3, p. 343-368. doi: 10.1080/00131911.2014.929565

Steer, D., Iverson, E. R., Egger, A. E., Kastens, K. A., Manduca, C. A., and McConnell, D. A., 2019, The InTeGrate materials development rubric: A framework and process for developing curricular materials that meet ambitious goals, in Gosselin, D. C., Egger, A. E., and Taber, J., eds., Interdisciplinary Teaching about Earth and the Environment for a Sustainable Future, Springer, p. 25-43.