TIDeS Curricular Materials Rubric

What does this mean?

• Through a holistic learning process, students will ask questions about a natural phenomenon or real-world engineering challenge, gather evidence to construct explanations or iteratively design engineering solutions, and communicate their reasoning to themselves and others.
• During experiences like these, students will make use of the science and engineering practices and crosscutting concepts in order to make sense of and deepen their understanding of core concepts and natural phenomena and to solve design challenges.

Examples and references:

Courses provide substantial opportunities to engage in the complete cycle of investigation and design throughout the curriculum.

For example during such a course, students could construct an initial model to explain a phenomenon, engage in learning activities that deepen their understanding of the phenomenon, revise their model, engage in activities to collect evidence for their revised model, and test their model with real-world data.

Reference: Chapters 4, 5, and 6 in Science and Engineering for Grades 6-12: Investigation and Design at the Centerfrom the National Academies of Science, Engineering, and Medicine

What does this mean?

• Multiple means of engagement, including multiple means of representing and expressing students' knowledge, are intentionally integrated, promoting equitable participation.
• Instruction builds on students' lived experiences, creating space to connect their experiences with core science or engineering topics.
• Instruction will have an emphasis on making meaning that includes students hearing and understanding the contributions of others, and communicating ideas in a common effort that builds understanding of natural phenomena or engineering solutions.

Examples and references:

Instructional materials include citations and references to materials from diverse authors and perspectives that represent as many student backgrounds as possible. Students should engage in activities that investigate scientific phenomena and engineering design challenges that relate to different contexts (e.g., urban, rural, and other types of communities). Students have opportunities to voice their perspective and points of view utilizing supportive data and evidence.

Reference: Equity-focused principles, strategies, and resources from the Center for Research on Learning and Teaching at University of Michigan

What does this mean?

• The phenomena and design challenges for students to investigate should be selected to be of interest to a wide variety of students as interest is proven to be a key catalyst for short- and long-term learning.
• Relevance can be established through personal interests, cultural contexts, real-world issues, and ongoing scientific questions, and the type of relevance should be varied throughout the course.
• Activities and materials connect phenomena and challenges in ways that cultivate student curiosity, build a sense of wonder, and allow students to make connections to their everyday experiences.
• Materials should the ways instructors can adapt or modify the phenomena and challenges to their own settings where possible.

Examples and references:

For example, students could investigate why a river floods. Activities could include materials (data, maps, visualizations, public records) related to a river that is regional to the classroom, allow for a field trip to a river (virtual or in-person), and provide an opportunity for students to practice their skills in investigation or design on a river of their choice.

What does this mean?

• A key aspect of investigating a phenomenon or design challenge is making sense of it through the collection and analysis of evidence involving the use of real data and models and the tools used by scientists and engineers.
• Course design should provide the structure needed to support students in developing their data collection, analysis, and interpretation skills (including computational and visualization skills) to build usable knowledge.
• In selecting the data, models, and scenarios, instructors should consider differences in access to technology, tools, and preparation of students. This could include having students collect their own data over time or using available data repositories (e.g., USGS, NEON, NASA)

Examples and references:

Students should use both tools for data collection and analysis—including spreadsheets, digital recording devices, etc...— and actual data that are used broadly by scientists and engineers in different fields.

Examples of data-rich activities includeProject EDDIE andTree-Ring Expeditions (TREX).

Recognize the limitations of activities that can be useful but do not reflect the actual practices: for example, mining a cupcake provides a fun visualization of core sampling but does not reflect authentic practices and has the potential to introduce misconceptions.

What does this mean?

• The materials should clearly communicate and describe intended unit- and course-level student learning outcomes.
• Goals and objectives should be stated in terms of what the student will be able to do when the work is completed, and be written as a statement that makes use of science and engineering practices and cross-cutting concepts to build understanding of disciplinary core ideas.
• Goals and objectives are substantial, measurable, and achievable (or plausible in the case of affective domain goals) by students at the end of a unit or course (as opposed to a single class period).

Goals will explicitly use concrete action verbs that correspond within various levels ofBloom's and Krathwohl's taxonomy for the cognitive, affective, and psychomotor domains.

Example performance expectation: Apply concepts of statistics and probability to support explanations that organisms with an advantageous heritable trait tend to increase in proportion to organisms lacking this trait. (NGSS HS-LS4-3)

Example affective domain outcome: [Reflect on} the inequity of climate change and the need for climate resilience in industrialized and developing countries. (Adapting to a changing world)

Reference: Read more aboutDesigning Measurable Learning Goals from InTeGrate

What does this mean?

• Learning objectives break down the learning goals into steps that are achievable in a single class period or activity.
• Scope, sequence, and how the objectives connect to the larger course goals should be evident. 
• Each objective links to previous objectives and provides a need to engage in the current objective. For cognitive-domain objectives, this is often achieved by moving through the levels of Bloom's taxonomy.

Like goals, the sequence of objectives should be explicit and use action verbs, and affective-domain objectives may not be as performance driven.This unit on Soil Characteristics from InTeGrate is guided by a set of objectives that build in complexity to connect to module-level goals. By the end of this unit, students will be able to:

• Describe the soil properties of porosity and permeability.
• Characterize the porosity and permeability of a soil sample.
• Interpret and assess the effects of land use practices on the porosity, permeability, and erosivity of the soil.
• Make recommendations for sustainable agricultural practices in a hypothetical scenario.

What does this mean?

• Across the course, many learning objectives should address specific skills in using data as evidence in explaining natural phenomena and making design decisions in written and/or oral format. 
• The data may be gathered from a reliable source or collected by students.

Examples come fromGETSI,InTeGrate, andProject EDDIE:

• [By the end of this unit, students will] communicate the probability of risk of volcanic eruption based on geodetic data to a non-expert. (Yellowstone is active, but is it erupting?)
• Use collected data to produce a map of sensory experience that conveys social and physical concerns clearly and accurately. (Sensory Map Development)
• Students will use data to compare short-term and long-term discharge variability, and quantify climate change impacts on water quantity in their region. (Stream Discharge Module)

What does this mean?

Goals and objectives:

• Set reasonable expectations for undergraduate students in introductory science courses, considering likely preparation in mathematics and related skills.
• Use asset-focused language. 
• Provide opportunities to develop and use specific elements in scientific investigation and engineering design.
• Are designed specifically with an appropriate scope and sequence and unit-level objectives map onto course-level goals.

The context for use section that accompanies the learning goals describes the intended audience and sets reasonable expectations using asset-focused language that emphasizes what students bring rather than what they lack.

Use asset-focused language to avoid labels such as "appropriate for majors or highly motivated students." Instead, for example: These learning objectives are appropriate for introductory environmental science, oceanography, or introductory geology or for a general introductory science course where the nature and methods of science are being investigated (Kortz & Smay). 

Reference: Read aboutAsset-Based Pedagogy at the Searle Center for Advancing Teaching and Learning at Northwestern.

What does this mean?

Learning objectives and goals are intentional, concise, detailed, and presented in language that is understandable by the audience, while promoting science literacy and an asset mindset that builds on the strengths students bring to the learning.

The context for use section and teaching tips section can identify expected prior knowledge to help explicate the level of the students, providing more information than their class year.

Units in theWater, Agriculture, and Sustainability module from InTeGrate provide information in the context for use section that further explains the expected student preparation. 

For example: Students can start the module with no shared preparation. Before each in-class activity of this first unit, each student will need to do the assigned readings, participate in the online discussions and (for unit 1.3) complete a homework assignment. This will give them the background necessary to analyze and critique the unit concepts and data.This unit can stand on its own, if desired. It is appropriate for college students at all levels and majors. It is of particular value in introducing Earth Science majors to the concept of sustainability and the roles of culture, politics, economics, and agriculture in the watery aspects of the Earth system.

What does this mean?

• Formative assessments provide opportunities to elicit student thinking and need not be graded or have any stakes associated with them. They provide feedback that can be used to guide improvements in the teaching and learning setting to help students succeed in achieving the learning goals. 
• Summative assessments provide logical tools to determine the extent to which students have met the learning objectives and goals, and should involve a substantial piece of student work.
• Together, formative and summative assessments reveal how students' thinking and problem-solving develops in a disciplinary domain and how students communicate their ideas.

Examples and references:

TheClimate change, after the storm unit from InTeGrate has examples of formative assessments and a substantial summative assessment. Formative assessments include collecting observations and questions on a chart, making concept maps, and discussing with peers. The summative assessment is a position paper in which studentsmake an argument from evidence to address the question,"To what extent should we build or rebuild coastal communities?

Reference:Read more aboutAssessments that align with your learning goals from InTeGrate

What does this mean?

• Assessments include a clear and meaningful list of criteria used to evaluate student work and participation, including the information students need to know about how a grade will be determined and the information another instructor would need to know in order to assess to what extent students have met learning objectives (their level of proficiency). 
• Levels of proficiency include criteria to assess student responses by different elements. 
• Rubrics for formative assessments may focus on "what to look/listen for" in student responses.

Examples and references:

Therubric for the position paper in Climate change, after the storm provides detailed levels of proficiency for elements of the paper. TheVALUE rubrics from AACU are well-tested and established rubrics for intellectual and practical skills (including written and oral communication) as well as other skills such as civic engagement. 

Be aware that rubrics with vague or overly general criteria can promote unintended biases (seeQuinn, 2020).

Reference: Read more about Teaching with rubrics from InTeGrate.

What does this mean?

• Materials should give students awareness of their thinking and provide multiple, varied, and iterative opportunities to elicit students' emerging ideas about a phenomenon or engineering challenge.
• Materials should include an appropriate balance of guidance versus exploration and include opportunities for reflection, discussion, and synthesis. 
• Materials should include formative means for evaluating how student thinking develops in the given disciplinary domain and take into account how students' lived experiences inform their understanding of the core ideas and crosscutting concepts. 
• Assessment methods for student discourse (e.g., observation of student discussion groups, whole-class discussion) should identify what practices and discursive behaviors would be expected at that point in the investigation and design process. 
• Additionally, materials should provide means for students to assess their own thinking and confirm they are on the right track.

Examples and references:

The InTeGrate moduleInteractions between water, Earth's surface, and human activity, has a description of how tosolicit students' initial ideas. Students can assess their own learning throughmetacognitive strategies such as think-pair-share and reflective prompts.

Recognize group dynamics that limit inclusion and instead establish class norms andstructured interactions) that create safe spaces for all to share ideas (see for exampleGrier-Reed & Williams-Wengerd, 2018). 

Reference:Discussion strategies from the Center for Teaching and Learning at Washington University in St. Louis.

What does this mean?

• Materials and activities should promote deep learning about a phenomenon or engineering challenge where the results of the investigation and/or design process are assessed. 
• A capstone, summative, or final product of the process will provide a means for students to demonstrate and to assess their understanding of the causes of a phenomena, or solution to challenges using evidence, constructing explanations, and being able to communicate their reasoning clearly to others.
• Effective summative assessments require a higher level of student thinking that synthesizes learning across concepts and/or domains. 

Examples and references:

A summative piece of student work can be used to evaluate student learning at the end of a unit or section of a course. Thecapstone project for the Earth's Thermostat module from InTeGrate provides an example, in which students develop and present a conceptual model exploring the possible climatic and societal effects of a Toba-scale volcanic eruption occurring in modern times.

What does this mean?

• Instructional strategies provide opportunities for students to build the skills and knowledge that will allow them to meet the expectations outlined in the objectives and goals and to engage in investigation and design.
• Throughout the course, specific strategies are likely to change as students gain more experience and confidence and instructional scaffolds can be removed or altered.
• Instructional strategies are inclusive, student-centered, evidence-based, and facilitate students' engagement in the practices of science and engineering.

Numerousevidence-based teaching strategies align well with investigation and design.Model-based inquiry describes an overarching framework for instructional strategies to plan for engagement, elicit student ideas, support students' changes in thinking, and pressing for evidence-based explanations. Strategies should involve supporting students in data collection, analysis, and interpretation. 

Reference:Instructional Scaffolding to Improve Learning from the Center for Innovative Teaching and Learning at Northern Illinois University

What does this mean?

• Strategies should allow students to practice scientific methods and engineering design processes with iterations to solve real world problems. 
• Opportunities provide students to take small bits of data and identify relationships to a larger, more encompassing set of ideas. 
• Relationships among ideas give ideas more credibility and help group big ideas into larger, more comprehensive ones to help students make sense of phenomena or design challenges.

Examples and references:

Chapter 5 How Teachers Support Investigation and Design describes specific strategies. Units can bedesigned so students explore first such as by observing patterns about a phenomenon as an opening activity, then introduced to concepts through discussion, video, or a virtual reality tour, followed by using data to analyze the phenomenon, incorporating their new learning and reflecting on it, as inthe Arctic fieldwork unit from the MOSAIC project.

What does this mean?

• Materials are designed to allow students to be curious and encourage students to ask questions, debate, and practice decision-making as an individual and as a team member. 
• Students should engage in productive science talk, providing exploration of ideas and use evidence to build and critique arguments, engage in peer collaboration, and communication, and feedback. Guidance for instructors in facilitating discourse is provided as needed.

Examples and references:

The instructional strategy could havestudents collaborate in small groups as they together answer questions on an activity sheet that guide them to a more in-depth conceptual understanding and then ask them to share their understanding with the rest of class. Instructors support discourse by using talk moves (such as those shown inTable 5-2 of the chapter How Teachers Support Investigation and Design).

Reference:Discourse Primer from Ambitious Science Teaching

What does this mean?

Activities include multiple opportunities for students to represent their emerging understanding surrounding a phenomena or design challenge through metacognitive prompts, models, or other learning artifacts.

Units could include metacognitive prompts that promote student reflection of the learning process such asone-minute papers. Additionally, reflective learning opportunities could be embedded in an iterative process where students buildmodels or another artifact that allows them to reflect and communicate their emergent understanding over time.

Reference: Read more aboutmetacognition and self-regulated learning from InTeGrate

What does this mean?

• Activities allow students to obtain, evaluate, and communicate information to become critical participants in the production and analysis of science. 
• Such activities provide opportunities to express, clarify, justify, interpret, and represent their ideas and be open to peer and teacher feedback orally/and or in written form.

Examples and references:

Different strategies encourage students to communicate, evaluate scientific arguments, and consider equity implications. Units could include having students present stakeholder positions in amock town hall or awall walk for a discussion of a controversial scientific topic orwriting a letter to a policy maker advocating for a particular design solution.

What does this mean?

• Instruction acknowledges students' different identities, experiences, strengths, and needs, and leverages student diversity as an asset for learning. 
• Strategies help students connect their prior knowledge or skills to new learning and give them a sense of belonging.

Examples and references:

When introducing a new topic, ask students to reflect on what they already know about the topic, or invite them to identify relevant skills they bring from different domains.

Highlight diverse contributions to the field, and give students the opportunity to explore and discover assets within their own communities and/or identity groups.

Reference:Academic  belongingfrom the Center for Research on Learning and Teaching at University of Michigan

What does this mean?

• Instructional resources and materials support instructors in helping students achieve the stated learning goals and objectives. 
• Materials contain explicit descriptions for instructors to help students access and build their skills in science and engineering practices to deepen their understanding in disciplinary concepts. 
• It should be evident that materials are unit-specific and focus on content, constructs, and principles that are related to the topics.

Examples and references:

Including instructor stories or teaching tips can describe how an activity could be adapted to engage students in a more regional, relevant example such as by theadaptations for a Coastal Hazards Risk Management plan activity or be augmented to strengthen students' conceptualization related to a learning objective such as through these teaching tips byreviewers of the Universities Space Research Association Catching a Heat Wave activity.

What does this mean?

• Western science relies on empirical and experiential knowing; other ways of knowing are personal, aesthetic, cultural. Instructional materials should recognize and highlight the funds of knowledge that students may bring from their own experiences and avoid use of words and phrases that privilege and/or assume a specific way of knowing. 
• Materials are culturally relevant and responsive to diverse learners. Cultural responsiveness builds on individual and cultural experiences and students' prior knowledge.

Examples and references:

Place-based education is one strategy for embedding other ways of knowing to understand physical features of landforms or ecosystems (seeSemken et al., 2018) orethnobotany orethnogeology, scientific fields that examine a human culture's knowledge in relation to the given scientific field.

What does this mean?

• The selection and language of materials and resources are aligned with the appropriate learner levels and learning styles. 
• Materials should reflect that contributions may be complicated by the social and political time period of those contributions. 
• Materials should reflect contributions from individuals with different identities and backgrounds. How those contributions have been historically valued differently should be included where possible.

Examples and references:

Materials should reflect diverse designers and acknowledge the specific contributions of members from multiple communities to scientific and engineering enterprises. Materials are justice-oriented and reflect the social context we're in now. Science and engineering topics should reflect current problems in engineering, but also reflect historical theories and principles when appropriate. See for exampleGeoContext: A social and political context for geoscience education.

What does this mean?

• Materials use data, resources, and questions/design challenges that are current and up-to-date at the time of publication. 
• Pictorial and descriptive images of STEM endeavors include images in historical, contemporary, and future-focused terms as appropriate. 
• Adherence to copyright permission should be explicitly practiced when using non-self-authored materials.

Examples and references:

Case studies such as theOso Landslide case study from GETSI provide opportunities to use up-to-date data and resources. Resources linked in this activity are attributed to their sources; many of the resources are in the public domain from the U.S. Geological Survey. Data are not subject to copyright, but the products produced from data, such as visualizations, likely are.

Reference:Copyright pointers for contributors to SERC-hosted sites

What does this mean?

• Accessibility of materials for both instructors and students is equitable. 
• Materials should follow principles of Universal Design for Learning, namely: multiple means of engagement, representation, and expression.
• Images have alt- tags and uploaded documents follow accessibility standards. 

Examples and references:

Web sites, national data repositories, software, scholarly articles, and other external resources are accessible to students and teachers; if not, alternative resources are provided for use. When needed, downloadable software/programs are free and available with minimal computer specifications.

Reference:Universal Design for Learning (UDL) Guidelines from CAST

What does this mean?

Each element of the curriculum within each section aligns with all other curricular elements (instructional strategies, activities, student materials, assessments) directly through the stated learning objectives and goals.

Examples and references:

Alignment across all elements should make it clear that the guiding principles of science and engineering design, equitable learning, problems that are relevant to student lives, and authentic scenarios that use real data are pervasive throughout the course.

What does this mean?

Curricular materials align directly with the stated course goals holistically across the entire curriculum.

Examples and references:

Alignment should demonstrate the coherent structure of the course in addressing the course goals.