EvaluateUR: Findings from the NSF Project
An award from the National Science Foundation's WIDER program supported the scaling up and dissemination of the EvaluateUR method. In 2019, the final year of NSF funding, EvaluateUR was implemented by undergraduate research directors at 37 institutions with 700 student-mentor research pairs completing all of the EvaluateUR steps.
Changes in assessment scores for this group and feedback from user surveys support a number of conclusions about the method:
- EvaluateURsucceeded in introducing student participants to a comprehensive list of competencies and skills they would need in order to go on to graduate work and/or succeed in the workplace.
- All EvaluateUR assessment components saw statistically significant positive growth for both students and mentors when tested by a paired sample t test (alpha =.05). Further, the Cohen's d effect size showed medium and large magnitudes of effect for almost all components for both mentors and students. The medium and large effect size suggest that the significance is not just due to chance or large sample size but to genuine impacts on student outcomes.
- Post-assessment student-mentor conversations contributed to the development and enhancement of student metacognitive skills, characterized by learners becoming aware of what learning strategies they are pursuing and why, and then using that awareness to make intentional adjustments to those strategies in order to learn more effectively. These conversations also helped most students confirm their plans for graduate school or employment in their major field.
- Research mentors found it easier to identify the academic strengths and weaknesses of the students they mentored, enabling them to focus their mentoring efforts more productively.
- The mentors could also more easily see areas where students might be over- or under- estimating their competencies and could help students gain new insights into their academic strengths and weaknesses and the relative efficacy of their learning strategies.
- EvaluateUR provided undergraduate research program directors with tools to help them manage their research program as well as provide them with reliable, readily understandable evidence of the potential benefits of their undergraduate research program.
Summary report of the findings from the 2019 implementation (Acrobat (PDF) 287kB Apr9 20)