Monitoring and measuring the development of earth education pathways: Strategies learned from the EarthConnections project
Monday
1:45pm
Ritchie Hall: 368
Oral Session Part of
Monday B: Earth Connections
Authors
Ellen Iverson, Carleton College
Karen Peterman, KAREN PETERMAN CONSULTING
Barbara Nagle, University of California-Berkeley
Mitchell Bender-Awalt, Carleton College
Cathy Manduca, Carleton College
A critical aspect of program assessment involves establishing a process for progress monitoring and measurement. Monitoring and measurement can identify early signs of success or unanticipated obstacles. Introducing structured practices for monitoring and measurement can enable team members to learn from each other, bolster a shared vision, and enhance collaborations and working relations. The EarthConnections: Community Pathways to Geoscience Careers project uses a collective impact approach to develop regionally focused earth education pathways. The design of a shared measurement system fulfills one of the five conditions of a collective impact approach. The EarthConnections shared measurement system includes three strategies: a discourse-centered checkpoint process, a web-based communications dashboard, and a set of common measures to document student outcomes. The EarthConnections metrics working group helped establish the processes and measures, gathering feedback from the EarthConnections steering committee and regional alliance members to formalize and adapt the processes. The checkpoint process allows regional alliance team members to routinely report on progress through a set of prompts that align to key areas of pathways development. Feedback is given through online discussion boards and a moderated virtual meeting involving all the regional alliance and steering committee members. A checkpoint visual rubric administered by the internal evaluator helps the alliance as a whole to monitor progress. Regional alliance members value the opportunity that the process provides for articulating bright spots and challenges, reflecting on progress, and learning through the rich discussion and feedback across the alliances. In addition, a real-time web analytic dashboard visually quantifies engagement metrics such as the numbers of email list messages and email list members and the visits to internal workspace pages. Lastly, the metrics working group identified a set of validated student measures that alliances will pilot at various points within their pathway to understand students' attitudes related to learning.