Workshop Instruments

End of workshop surveys

The end of workshop survey is the most critical evaluation instrument in measuring the success of the workshop. It is important to have 100% participation and collect a survey from each participant before they leave the workshop. Both a paper template (Microsoft Word 102kB Feb22 10) and an online version of the template are available for you to customize for your workshop. You need to choose the method (or combination) that can best facilitate full participation. If you choose to use the paper template, you will need to print each form as a mail merge with unique identifiers in the header.

Action plans

After repeatedly hearing from participants that they would like more reflection time, we instituted an action plan activity at the close of the workshop. Developing an action plan gives participants the time to reflect on how they'll put what they learned to practice as well as gives you an idea of the sustained impact of your workshop. You can choose to have participants complete a highly structured action plan structured action plan (Microsoft Word 71kB Jan15 09) or include action plan questions in your end of workshop survey (see questions 7 and 8 in the paper template (Microsoft Word 102kB Feb22 10)).

Roadchecks

For multiple day workshops, we collect formative responses from participants at the end of each workshop day through a roadcheck form. The roadcheck gives conveners immediate feedback on how the program is going for participants. It can also provide input it into the structure of the remaining workshop days (if that flexibility is possible). Roadchecks should be anonymous unless it is necessary to collect summative or longitudinal data.

Embedded Assessment

We hope to collect evaluation data that is more than self-report from participants. To this aim, we include an embedded assessment in many of the Cutting Edge workshops. These assessments involve a teaching activity review or a reflective professional development piece.

Teaching Activity Review

Prior to the workshop all participants submit at least one activity or assignment for actively engaging students in the classroom or for providing effective and innovative lab or field experiences. As part of the workshop program participants review each other's activities. Two approaches have been used in past workshops:

Review approach
You can use a set of questions to guide the review of the assignment or activity. You may also precede this with a presentation such as this one, by Cathy Manduca: What Makes a Good Activity? Lessons from Research and Experience (PowerPoint 463kB Oct22 08).

Rubric approach
You can introduce a rubric (Acrobat (PDF) 18kB Jul16 08) for evaluation of the quality of an assignment or activity. Participants consider strategies for improving the effectiveness of a sample activity in preparation for reviewing each other's activities. Participants then use a slightly different rubric (Acrobat (PDF) 30kB Jul16 08) to review and provide comments on activities that have been submitted for this workshop and provide feedback to the authors. You may precede this with a presentation such as this one, by Barbara Tewksbury: Designing Effective Assignments and Activities (PowerPoint 651kB Jul16 08).

With either approach the aim of this session is two-fold:

1. to encourage high quality contributions to the workshop website and
2. to help participants think critically about what makes a good teaching activity.

Both the application for the workshop and the end of workshop survey should include the question: What are the features that you look for in a strong teaching activity? The Cutting Edge evaluation team will be analyzing changes (pre-workshop/post-workshop) in responses to these questions, in aggregate.

Reflective Professional Development Piece

For workshops where participants are just beginning, or about to begin, their academic careers, we often substitute a reflective professional development piece for the teaching activity review described above. Here are two such activities:

Elevator Talk

Participants in the Cutting Edge workshop on Preparing for an Academic Career in the Geosciences are asked to write an elevator talk prior to the workshop. Here's how it's described:

In the job search and interview process participants preparing for a career will have very brief, yet critical, opportunities to convey their work to departmental faculty, institutional administrators, and students. In preparation for that, we ask participants to draft a BRIEF introduction of themselves and their research. The goal is to convey the nature and significance of their work to a scientist (but not necessarily a geoscientist).

As part of the workshop program, participants have an opportunity to practice their elevator talk, get feedback, and make revisions to their statement. The Cutting Edge evaluation team analyzes changes (pre-workshop/post-workshop) in these elevator talk statements.

Poster about Teaching and Research

As part of the workshop for Early Career Geoscience Faculty members, participants create a poster that describes 1) a teaching activity, assignment, or course and 2) their plan for a research or scholarly activity. Participants present their posters and receive feedback from others on both of these components. This poster session is a culmination of the dual focus of the workshop program on best teaching practices and research strategies. Following this session participants are asked to reflect on the ideas from their poster, how the workshop has influenced these ideas, and how the ideas will change as a result of the feedback. They record their thoughts on this poster reflection form (Microsoft Word 31kB Jan15 09).