- Scheduling the orientation session
- Introducing EvaluateUR and its Structure
- Student-Mentor Conversations
- Automated Messages
- Tips for Tracking Student and Mentor Progress
- Timeline/Flowchart for Completing Steps
To ensure that the participants in Evaluate UR are familiar with the program's method and design, site administrators should provide an orientation session for student researchers and their faculty mentors, before research begins. Feedback from site administrators who have implemented EvaluateUR has stressed the importance of having all program participants attend the same orientation session. This will increase the likelihood that EvaluateUR will be implemented successfully, since the session will help students and mentors sharpen their understanding of the value of the program's methods as well as the sequence of steps to be completed. In any case, to ensure that both student and mentor members of your program's research pairs receive the same information, every research pair should be required to attend the orientation session together. Experience shows that attendance will be enhanced if the session is scheduled to avoid finals week, but more than one orientation session may be necessary in order to accommodate varied student and mentor schedules. Site administrators should plan to spend a minimum of 60 minutes introducing EvaluateUR and answering questions.
Consider beginning the orientation session by providing students and mentors with some background about the development of EvaluateUR and how this approach can be used to strengthen the research experience and benefit both students and mentors, as well as the undergraduate research program at their institutions. The Roadmap includes a discussion of this topic (see the section called "Benefits to Students, Mentors and Administrators"). Also, the Publications page includes links to articles published in the CUR Quarterly that describe the development of EvaluateUR, and discuss findings based on many years of its implementation at SUNY Buffalo State. You might also want to reassure students and mentors about the level of effort the program requires. Experience indicates that EvaluateUR generally requires several hours over the duration of the research experience and that the conversations that follow each of the three assessments complement (but do not replace) regularly scheduled student/mentor meetings.
Orientation Session Slideset (PowerPoint 2007 (.pptx) 2.3MB Apr16 20)
It is very important to emphasize the purpose of the student-mentor conversations. Key points include:
- Assessment scores given by mentors are not 'grades.'
- Student self-assessment scores do not need to agree with mentor assessment scores.
- The objective is not agreement, but a conversation that can uncover differences in perceptions, assumptions, and reasoning.
For both students and mentors the protocol for these conversations should emphasize the importance of:
- Careful regard for evidence to support one's views
- Open-minded and non-defensive listening
- A spirit of collaboration in the service of understanding and problem solving
- Consideration of what would be needed in order to resolve important differences in assessment scores
- Where students and mentor agree that improvement is needed, a focus on strategies that could help to make that happen
You might find it useful to distribute the list of 11 outcome categories and their components along with the scoring rubric. We also recommend that you provide a handout illustrating the Dashboard and showing the sequence and timing of steps to be completed.
Handouts_for_Orientation_Session.docx (Microsoft Word 2007 (.docx) 311kB Apr16 20)
It should be pointed out to students and mentors that they are likely to receive automated messages after the completion of certain steps in the program. Usually the message just confirms that a form was properly submitted. Other messages are sent to remind students and mentors about next steps requiring their attention, or as a reminder to complete a step if they miss a deadline. We have found that automating these messages is an efficient way to accomplish an important administrative task while freeing site administrators to focus on more complicated issues. Most messages are sent during the beginning, middle, and end of the research program, when student-mentor assessments and conversations take place.
The fidelity of this method requires the completion of the steps at the proper time in the research experience. Furthermore, EZStats will only provide data for those pairs that complete all three assessments.
Site administrators should note dates for their program benchmarks (start, middle, and end). These dates can be the same for all participants in the program or can be adjusted for each student-mentor pair when programs have varying beginning and end dates for participants. Based on progress, the dates can be adjusted as needed.
The following timeline identifies EvaluateUR steps for each research phase.