Assessing Geoscience Programs: Theory and Practice

Workshop Program

Jump down to Tuesday

Sunday, February 22

5:00 Opening Reception and Dinner at the Alumni Guest House Meeting Room & Library, Carleton College

7:30 Welcome, Workshop Goals and Overview - Cathy Manduca, Science Education Resource Center, Carleton College

Opening Keynote on Program Assessment: What Difference Does it Make? (PowerPoint 657kB Feb22 09) - Ross Miller, Association of American Colleges and Universities

Show related resources

Hide

• Research Associated with Greater Expectations, LEAP, College Learning for the New Global Century, and Other Formative Educational Ideas (Microsoft Word 63kB Feb20 09) from Ross Miller
• On the Cutting Edge website on Student Motivations and Attitudes: the Role of the Affective Domain in Geoscience Learning
• On the Cutting Edge website on the Role of Metacognition in Teaching Geoscience

Discussion: Making the Most of Program Assessment

Monday, February 23

Breakfast at the Country Inn

All workshop activities will take place at the Alumni Guest House Meeting Room, unless noted.

8:45 Opening Remarks

9:00-10:30 Three examples:

• Sheila Roberts (Bowling Green State University) - Assessing Graduate and Undergraduate Geoscience Programs: Similarities and Contrasts (PowerPoint 840kB Feb20 09)
• Michael Zieg (Slippery Rock University) - Maintaining Coherence: Integration of Course, Program, and University Outcomes (PowerPoint 4.2MB Feb22 09)
• Allen McGrew (University of Dayton) - Designing an Assessment Process (PowerPoint 4.2MB Feb23 09)

10:30-11:00 Break

11:00-12:00 Small Group Discussion: What do we know about what works in creating plans?

Each group will discuss their own experiences, as well as the things they have learned from the essays, and reflect on what suggestions we would give to others.

Create a bulleted list of effective management practices. These could address setting goals, developing the plan, getting buy in, or implementation. You may want to use cautionary tales from the essays to anticipate things that should go on this list. Please upload your list and be prepared to report out in a round robin discussion.

• Recommendations from Group 1: (Microsoft Word 29kB Feb23 09) Dale Sawyer, Darrell Henry, Sheila Roberts, Ann Bykerk-Kauffman, Michael Zieg
• Recommendations from Group 2: (Microsoft Word 27kB Feb23 09) Cinzia Cervato, Gudrun Willett, Bill Hart, Karen Rose Cercone, Ross Miller
• Recommendations from Group 3: (Microsoft Word 21kB Feb23 09) Karen Viskupic, DaveDempsey, Al McGrew, Bernie Saini-Eidukat, Heather Macdonald
• Recommendations from Group 4: (Microsoft Word 19kB Feb24 09) Charlotte Mehrtens, George Kipphut, Bev Johnson, John Bartley, Sue McGeary
• Recommendations from Group 5: (Microsoft Word 24kB Feb23 09) Cathy Summa, Steve Lundblad, Margaret Boorstein, Mark Hemphill-Haley
• Recommendations from Group 6: (Microsoft Word 25kB Feb23 09) Dirk Baron, Mary Savina, Leila Gonzales, Arlo Weil, John Bartley
• Recommendations from Group 7: (Microsoft Word 37kB Feb23 09) Eric Pyle, Eric Peterson, Steve Moshier, Nelson Ham, LeeAnn Srogi

12:00-12:30 Whole Group Synthesis of Recommendations

12:30-2:00 Lunch at Burton Cafeteria

2:00-3:30 Four Examples

• Eric Pyle (James Madison University) - Developing and Implementing a Program-Wide Student Performance Assessment Protocol: Preliminary Results (Acrobat (PDF) 1.5MB Feb23 09)
• Dave Dempsey (San Francisco State University) - The Promise and Challenges of Portfolio Assessment (PowerPoint 577kB Feb23 09)
• William Hart (Miami University) - Assessing Capstone Experiences: An Example from Field Methods (PowerPoint 2.2MB Feb23 09)
• Darrell Henry (Louisiana State University) - Rubrics in a Communications-Intensive Geology-Majors Course: Their Roles in Departmental Assessment (PowerPoint 637kB Feb22 09)

Show related resources

Hide

• LSU's Communication Across the Curriculum Website
• Darrell's Igneous and Metamorphic Petrology Course Website

3:30-4:00 Break

4:00-5:00 Topical Breakout Groups

Discuss under what circumstances this technique is useful, what kind of questions does this tool answer best, what leads to a successful implementation, and where the challenges lie. Produce a list of findings/recommendations (these could be very specific) and outstanding questions. Upload your list of recommendations. Be prepared to briefly share your most important point.

Topics:

• Exit Surveys/Senior Interviews/Student Surveys (Microsoft Word 46kB Feb23 09): George Kipphut, Lee Ann Srogi, Cathy Summa, Char Mehrtens, Cinzia Cervato, Heather Macdonald
• Alumni Surveys (Microsoft Word 32kB Feb23 09): Steve Lundblad, Leila Gonzales, Nelson Ham, Dirk Baron
• Portfolios (Microsoft Word 2007 (.docx) 12kB Feb23 09): Al McGrew, Karen Rose Cercone, Dave Dempsey, Steve Moshier, Dale Sawyer, Sheila Roberts
• Classroom Assessments (Microsoft Word 41kB Feb23 09): John Bartley, Darrell Henry, Bev Johnson, Margaret Boorstein, Ann Bykerk-Kauffman, Sue McGeary, Bernhardt Saini-Eidukat
• Research Programs/Capstones Experiences (Microsoft Word 22kB Feb23 09): Michael Zieg, Arlo Weil, Eric Peterson, Cam Davidson, Karen Viskupic
• Assessing Field Work (Microsoft Word 30kB Feb23 09): Mark Hemphill-Haley, Bill Hart, John Geissman, Eric Pyle

5:00-6:30 Making Sense of What We Have Learned

• Brief Report Outs
• Thinking about the Big Picture: Lessons from Program Assessment (PowerPoint 1.5MB Feb24 09) - Cathy Manduca, Science Education Resource Center, Carleton College
• Individual Reflection
• Thinking About Tomorrow

Dinner - Explore Northfield

Tuesday, February 24

Breakfast at the Country Inn

8:45-9:00 Opening remarks

9:00-10:30 Presentation, panel and discussion: What Makes Reviews Work? Considerations from the reviewer and reviewee points of view

• Department Reviews (PowerPoint 950kB Feb23 09) - Mary Savina, Carleton College
• 10-year Review Self-Study (PowerPoint 95kB Feb24 09) - Steve Moshier, Wheaton College
• Program Review: Experiences from Within and Without (PowerPoint 366kB Feb23 09) - Char Mehrtens, University of Vermont

Read the departmental review discussion summary

10:30-10:45 Break

10:45-11:45 Designing reviews to address specific goals/issues

Small group activity and reporting out: On your assigned topic(s) discuss what are critical or common problems and questions in this area? How might the departmental review be used to help the department clarify or solve some of these: What would you want to include in the self study (data, comparisons, arguments)? How would you obtain this data? Would you want to ask the external reviewer for some specific comment (if so what)? What might you anticipate that the external reviewer might want to ask? In sum this discussion should provide the guidance that a department needs to successfully complete or address this topic through the review.

For each topic upload your recommendations for effective departmental reviews via this form.

Topics:

• Curriculum (Microsoft Word 40kB Feb26 09): Dale Sawyer, Cinzia Cervato, Al McGrew
• Learning outcomes (Microsoft Word 27kB Feb26 09): Sheila Roberts, Darrell Henry, Eric Pyle, Margaret Boorstein
• Personnel (Microsoft Word 29kB Feb24 09): Gudrun Willet, John Bartley, Dirk Baron
• Facilities (Microsoft Word 26kB Feb24 09): Nelson Ham and Stephen Moshier
• Department Productivity and Metrics (Microsoft Word 38kB Feb24 09): Ann Bykerk-Kauffman, Leila Gonzales, LeeAnn Srogi
• Faculty Scholarship and Teaching in Department Reviews (Microsoft Word 28kB Feb24 09): Eric Peterson, Dave Dempsey, Karen Viskupic
• Student and Student/Faculty Research (Microsoft Word 48kB Feb24 09) Bill Hart, Bernie Saini-Eidukat, Steve Lundblad
• Faculty Workload Equity (Microsoft Word 31kB Feb24 09): Sue McGeary, Mark Hemphill-Haley, Karen Rose Cercone
• Position of department within institution/respect for department (Microsoft Word 27kB Feb24 09): John Geissman, Bev Johnson, Michael Zieg, Cathy Summa
• Using Alumni in Program Reviews (Microsoft Word 36kB Feb24 09): George Kipphut, Arlo Weil, Char Mehrtens

11:45-12:15 Departmental Reviews, Institutional Accreditation, and Discipline-Specific Accreditation Discussion

12:15-12:30 Develop Working Groups

12:30-1:30 Lunch at Burton Cafeteria

1:45-2:30 Grades and Assessment

2:30-4:30 Working Groups (including break)

Working Group Summary Documents:

• Alumni Surveys (Microsoft Word 23kB Feb24 09) and Alumni Survey Questions (Excel 25kB Feb24 09): Dirk Baron, Nelson Ham, Cathy Summa, Leila Gonzales
• Categories of Questions for Exit Surveys (Microsoft Word 49kB May13 09): Bill Hart, George Kipphut, Al McGrew, LeeAnn Srogi
• Examples of Undergraduate Learning Goals and Associated Student Learning Outcomes (Microsoft Word 34kB Feb24 09): John Bartley, Mark Hemphill-Haley, Eric Peterson, Dale Sawyer, Cinzia Cervato, Michael Zieg, Sue McGeary
• Logic Model for Assessing a Program Goal (PowerPoint 64kB Feb24 09) and Concept Map of an Idealized Assessment Cycle (Acrobat (PDF) PRIVATE FILE 438kB Feb11 09): Bev Johnson, Steve Lundblad, Bernie Saini-Eidukat, Margaret Boorstein, Dave Dempsey, Steve Moshier
• Rubric for Evaluating a Capstone Experience (Microsoft Word 51kB Feb24 09): Arlo Weil, Cam Davidson, Char Mehrtens, Karen Viskupic
• Draft Rubric for Evaluating Field Notes (Excel 34kB Feb24 09): Sheila Roberts, Darrell Henry, Ann Bykerk-Kauffman, John Geissman, Karen Rose Cercone, Eric Pyle

Upload your working group's summary

4:30-5:30 Discussion and Synthesis: Promising practices

Read a summary of the synthesis discussion: lessons learned and next steps

5:30-6:30 Next steps, action plans, evaluation

6:30 Final Dinner at Chapati