Frequently Asked Questions

Jump to the following Questions:

Technical issues

What if a student/mentor doesn't receive an email?
How can I keep automated messages from being marked as 'junk' or 'spam'?
What if I need to change/correct a student or mentor email address?
Program Administrators: How do I know if students and mentors are on track?
How long is the data stored and accessible? When does the period of storage begin?

Setting up EvaluateUR

Can a single mentor be paired with multiple students?
Who can be a mentor for the EvaluateUR method?
How should I determine the deadlines to use for the default dates?

Scoring assessments

The 5-point scale does not seem to provide a wide enough range?
Is it realistic to assign a score of 5 because that would indicate that the outcome components would always be observed?

What if a student/mentor doesn't receive an email?

Response:

Confirm that they were supposed to receive an email. Note: a mentor will not receive any email from the system until the student has submitted their reflection. See Mentor Feedback. You can also use the 'Preview All Steps' link at the top of your dashboard to see all the messages students and mentors should receive at each step of the process.
Make sure the email address entered into the system is the one the student/mentor is actually checking. Check for any possible typos in the information you entered and cases where people have multiple email addresses and don't regularly check the one you used.
Ask the student/mentor to check their spam folder. Some campus email systems are especially aggressive about marking messages as spam. We strongly encourage you to get in touch with your campus IT staff before you start using EvaluateUR and have them 'whitelist' all EvaluateUR related email to ensure it doesn't end up in a spam folder. Learn more about how to whitelist emails for individuals and institutions
You can see the timing and content of the emails already sent to mentors and students using the 'Show Emails Already Sent' option available in the expanded view of each pair. In cases where the email message refers the student/mentor to a url, it's the same url in all cases. In cases where an email wasn't received you could simply provide the same information manually with a direct email to the student or mentor. But in most cases it's worth trouble-shooting a bit more to make sure the underlying problem or source of confusion has been addressed. That ensures that there won't be issues with missing emails later in the process.

How can I keep automated messages from being marked as 'junk' or 'spam'?

Some campus email systems are especially aggressive about marking messages as spam. We strongly encourage you to get in touch with your campus IT folks before you start using EvaluateUR and have them 'whitelist' all EvaluateUR related email to ensure it doesn't end up in a spam folder. Learn more about how to whitelist emails for individuals and institutions

Question: What if a student or mentor needs to change their email address?

Response: If the pair has not yet been activated you can simply change and save the new address in the listing for their pair in the Dashboard (the site administrator's EvaluateUR home page). After the pair has been activated the student and mentor will each have an account and only they will be able to change the email address and name for their account. To do this they would:

Visit the Account/Favorites menu in the top right corner of the page and select the top entry which is for their account.
On the subsequent page which allows them to manage their account there is an option in the upper right to 'change email'. Selecting this will walk the individual through the process. Similarly there is a 'Edit Personal Information' link in the lower left corner their Account page to where they can correct their name.
After completing this the corrected information will automatically get used for future automated messages. If they have changed their email address the updated one will serve as the address they need to use to log back into their EvaluateUR account.

Show More Detail:

If the site administrator enters an email address incorrectly (e.g., one with a typo) the student/mentor will (obviously) not receive the initial email instructing them to start the process. In this case you'll need to notice this situation. If the issue is with the student address you can simply delete the pair (using the option on the right of you expand the details about the pair) and re-enter and reactivate the pair to start over. If the issue is with the mentor email and the student has already started you won't be able to delete the pair. Instead contact the mentor and let them know they'll need to go to the student/mentor page (provide them with that url), enter the incorrect email address to create their initial account and then go through the procedure above to correct the address.

Question: How should I determine the deadlines to use for the default dates?

Response: The dashboard allows the site administrator to set deadlines for completion of each of the assessments. The default deadlines must be entered before activating any of your pairs. The default dates are determined by the site administrator and based on the length and timing of your program. This allows flexibility as it accommodates research experiences of varying lengths.

Show More Detail:

The deadlines are a tool for the site administrator to more easily see which research pairs need a reminder about completing a step in the process. In general, students and mentors should complete each step immediately after they receive the automated message that the next step is available and requires their attention within the next day or two. The deadline system is a way for the site administrator to see which pairs are not meeting this requirement. When a pair misses a deadline the site administrator is sent an email and the pair is highlighted on the Dashboard. This should serve to remind the site administrator to contact the pair directly to find out what their situation is and to get them back on track. Site administrators will also receive a summary one a week identifying all pairs that have missed a deadline.

While deadlines mark the point at which pairs should have completed each assessment, the point at which pairs start each assessment is manually controlled by the administrator. For the initial assessment the pair does the pre-survey steps and then is at step 3 "waiting for release of the initial assessment". At that point the administrator has an option in the dashboard to release the assessment to each pair at whatever time makes sense given their individual schedules (which may be at the same time for all pairs or not). For the mid and final assessment, the mentor will be automatically emailed exactly 1 week before the deadline and asked to fill out a form confirming they are on schedule (or offering an explanation if they are not). Once they fill out that form the pair will be in a state where the administrator can release the assessment to them from the dashboard. If their schedule has shifted for some reason the administrator can work with them to adjust their schedule: potentially shifting the deadline date for that pair and moving them back to the previous step, or just holding off on releasing their access to the assessment until an appropriate moment.

Question: How do I know if students and mentors are on track and filling out assessments on schedule?

Response: Administrators are not copied on emails reminding students and mentors to fill out assessments. Instead administrators can check the dashboard anytime to see the current status of any pair. In the dashboard they can see the content of all emails sent and when they were sent to each individual. And when any pair falls behind (and therefore the administrator would need to take action) the administrator is emailed about the issue. With this system the administrator doesn't have see every email sent and somehow keep track of who might be behind. Instead the dashboard is tracking that for them and letting them know when problems actually arise.

Question: How long is data stored and accessible? When does the storage period begin?

Response: EvaluateUR data is stored for 1 year. The one year storage starts from the date where the first pair is activated (given access to start).

For more on our data storage practices read about Data Security and IRB.

Question: The 5-point scale does not seem to provide a wide enough range?

Response: A wider range of the scale could result in students and mentors becoming more concerned about assigning a score to a particular outcome component. The difference between two scores would become smaller if the scale were increased. While EvaluateUR does generate summary statistics based on the scores assigned by students and mentors, a strength of the method is the repeated structured conversations between the student and mentor and the feedback that the student receives.

Show More Detail:

EvaluateUR is not about having students and mentors 'agree' on the score they assign, nor is it about scores increasing or decreasing over time. The value of the EvaluateUR method is the conversation that takes place during which the student and mentor talk about the reasons why a particular score was given for a component. It might be that the student and mentor assign the same score but for very different reasons; or, they might assign different scores but discover that they really are not very different in their observations and rationale for assigning a score.

Question: Is it realistic to assign a score of 5 (always) because that means 100% of the time and next to impossible to achieve?

Response: If a student or mentor is uncomfortable assigning a '5' for that reason one option is to suggest that '5' means 'almost always' or else assign a '4' (usually). In the conversation, it should become evident to the student and mentor what each means by a '4' versus a '5'.

Can a single mentor be paired with multiple students?

Response: Yes. As the site administrator, you can enter the same mentor information in each pair and when you activate the pairs, the system will link all the individual students to that mentor's dashboard. When the mentor visits their site, the system will only be showing them one student at a time. When the mentor first visits (e.g., by following a link in an email message), the system will prompt them to choose which of their students to look at. There is also an interface in the upper right corner for them to see which student they are currently looking at and to switch their view to any of their other students.

Note: The menus to choose between students only includes students that have started filling out the initial reflection. Students that have not started the process will not show up for the mentor.

Who can be a mentor for the EvaluateUR method?

Response: A mentor should be the person that is most directly interacting with the student during their research. For example, Ph.D. students, post-docs, and researcher associates are often involved in mentoring undergraduates conducting research and certainly can be identified as the mentor in EvaluateUR.

« Previous Page