Design your Evaluation
An Example: Requirements of all Cutting Edge workshops
- Collect an individual action plan and end of workshop survey from each participant (see workshop instruments for more information)
- Complete a post-workshop follow up survey of participants that focuses on action plan responses
Overall philosophy of Cutting Edge evaluation
We hope that all workshop evaluation provides both formative input for the conveners and also supports the summative goals of the Cutting Edge program. Our summative evaluation focuses on understanding how the program impacts geoscience faculty, community, and culture. A second aim is to evaluate how the program contributes to the research base on effective faculty development.
To support both the formative and summative goals, the templates for the workshop instruments are designed to 1) give us a standard set of data about the workshop effect on participants and 2) give you valuable feedback as a convener.
Sensitivity to Data
In order to have evaluation data that is longitudinally linked to a given participant, we request participants to include their name. For example, we collect responses to the question "What are the features that you look for in a strong teaching activity?" in the application and the end of workshop survey. For the end of workshop survey we indicate that their responses will be kept confidential and that the evaluator will remove their name and only use it to make this linkage. We indicate that the project leaders will not see their name. For these instruments the data must first go to the the Cutting Edge evaluation team and will be provided to the conveners in aggregate.
For workshop surveys where names are not necessary (such as formative assessments, known as 'daily roadchecks') the data can be made immediately available to the designated evaluation contact for the workshop convener team. However, this contact needs to be sensitive to the verbatim responses in how the data is shared with the workshop team.
Establishing the Evaluation Plan
In planning for your evaluation of the workshop think about:
- What do you want the overall outcome(s) of the workshop to be for the participants?
- How would you define or recognize success?
- What do you want the larger community to learn from the workshop (via the website)?
We routinely use three types of surveys (daily roadchecks, action plans, and end of workshop surveys) as well as the workshop application (for pre-workshop measures) in Cutting Edge workshop evaluation. These are designed to measure how well the workshop executes explicit goals and the value and effect on its participants.
Critical to your plan is allowing time for participants to complete the evaluation instruments. Cutting Edge workshops aim to have 100% participation with the instruments. You'll need to include time in the program for the evaluation and consider whether paper or online versions of the forms can best facilitate this participation. It is nearly impossible to get participants to complete these surveys after they have left the workshop.