To begin EvaluateUR, the site administrator will need to open their institution's main page and register each student-mentor pair on the Institution Registration page of EvaluateUR:
- Set the default deadlines at the bottom of the registration page.
- Enter the information about each student-mentor pair.
It is critical to enter correct email addresses (watch for typos!) and to ensure that each email address is the one used regularly by the individual. Accuracy is essential because these will be the email addresses where all automated messages are sent. Once a pair is 'activated' to begin their collaboration in the EvaluateUR program, you will not be able to correct email addresses yourself. The student or mentor will have to make any necessary changes.
Before moving on to activate each student-mentor pair, adapt to your own style the template message and send the message to each student in the program. If you wish, you may copy the faculty mentor on this message. Then activate each pair when you are ready to have them begin to implement EvaluateUR. Activating a pair will immediately send an initial message to the students instructing them to create their account and complete the Pre-Research Reflection.
The Student Reflection asks students to provide (online) short responses to a series of open-ended questions seeking their views about the research process and how they approach and think about topics related to research. The reflection is designed to introduce the student to their research mentor and can be done any time after the student is accepted into the program and prior to the orientation session attended by the student-mentor pair. While this is especially useful for REU and other similar programs in which students and mentors have had no (or minimal) prior interactions, it is valuable for all student-mentor pairs, as it is unlikely that students have previously shared all their ideas about research, including any concerns they might have.
Depending on the level of detail provided by the student, the Reflection should take about an hour to complete. There is a 'save' function at the bottom of the form, so students have the option to begin answering the questionnaire, then completing it at a later time. Instructions to the students on when to complete the questionnaire are up to site administrators, but it would be best if the questionnaires were completed as soon as possible and at least two weeks prior to the scheduled orientation session. This will allow time for mentors to review the responses and contact the students.
Once the students complete the reflection, they select the 'submit' button at the bottom of the form. Once submitted, an automated message is sent to the mentors instructing them to set up an account then access and review the student's responses. To acknowledge the review of the reflection an automated message is sent to the student using the template response. Mentors have the ability to edit this message to provide specific feedback, request more detail, or even suggest that the student review materials in preparation for starting the research project. The mentor's message is sent to the student when they use the 'message to student' button at the bottom of the form. Further guidance about this step that administrators can share with mentors can be found in the Resources section.
To ensure that all participants in the program are familiar with the EvaluateUR method and its design, the site administrator should schedule an orientation session to be held prior to the start of the research program. The site administrator should also make certain, before the session begins, that students have completed their reflections and that mentors have reviewed the reflections and responded accordingly to the students. To ensure that both students and mentors in each research pair receive the same information, both members of each pair should attend the same orientation session. A guide for planning and running a program orientation session is provided in the Resources section. Information provided in the guide is organized into sections covering: 1) scheduling the orientation session; 2) introducing students and mentors to EvaluateUR and its structure; 3) student-mentor discussions regarding the pre-, mid-, and end-of-research assessments; 4) handouts describing the Dashboard, student outcomes and outcome components to be assessed; 5) automated messages; 6) tips for tracking student/mentor progress; and 7) the sequence and timing of EvaluateUR steps. Orientation Guide
- Student and Mentor meet to discuss all outcome components and assign scores, identify and score additional outcomes, and submit assessments
- Student and Mentor use URL link in automated message to access Score Report showing outcome ratings
- Student and Mentor discuss Score Report
In this step, the student/mentor pairs schedule a block of time when they can meet in order to complete the pre-research assessment. This is the first of three assessments, but because the research has not yet begun, the procedure for this assessment differs from that of the assessments that take place at the mid-point and end of the research. The sequence in this meeting involves discussing each outcome category and the components used to define the category. At this first meeting, the student and mentor discuss the 11 outcome categories and components within each one after which they take a few minutes to individually enter a score for each outcome component. After completing this for the 11 outcome categories, the student-mentor pair have the ability to add, discuss, and score several additional outcome categories and/or components to their assessment forms. Once both submit their assessment, an email is generated automatically that includes a link to a Score Report listing all of the outcome components organized by outcome category, together with columns showing both the student and mentor scores. Any outcome component with a rating difference of 2 or more is highlighted. Before ending the meeting, the student and mentor should review the Score Report and discuss any component with a difference of 2 or more.
The duration of this conversation will vary among student/mentor pairs. We recommend that 90 minutes, on average, should be set aside; less time likely is needed in situations where the student and mentor have worked together before. This meeting could coincide with the time students and mentors spend discussing research plans.
This step should be described with particular care during the orientation, in particular the way in which the procedure differs from the procedure for the mid- and end-of-research assessments.