University of Oklahoma
The University of Oklahoma (OU) started their change work in 2019, prior to their involvement in the HHMI grant, with a grassroots effort among faculty who were dissatisfied with the quality of feedback they received on their teaching. The end-of-semester course evaluation survey items were a particular pain point for faculty, as the questions had not changed for a few decades and weren't providing useful feedback for faculty to improve their teaching. The old survey also made it "too easy" to replace thoughtful evaluation of teaching with averages from the old-style surveys. Unit leaders could just grab a number from the survey and then administrators could use that number to rank people. Faculty members conducted research into literature confirming that this is not a valid way to evaluate effective teaching. While the policy at OU made the recommendation not to use student survey numbers as the sole source for teaching evaluation, anecdotal evidence, and, later, a faculty survey, indicated that at least some units were doing exactly that.
The faculty who were vocal about these issues were invited to join a committee that eventually became known as the Teaching Evaluation Working Group (TEWG). The TEWG was charged with providing evidence-informed advice and resources for departments wishing to update their teaching evaluation practices. The TEWG agreed to tackle the creation and piloting of a new student course survey as well. The new survey, called the Course Reflection Survey (CRS), is an instrument designed to provide more meaningful and useful feedback for instructors, as well as to yield useful information for administrators to evaluate teaching. The questions target the student's experience in the course and ask concrete questions rather than asking students to evaluate the instructor. The CRS was piloted and assessed between 2020 and 2022, when it became the official course survey instrument at OU.
The TEWG intentionally invited faculty from units that were resistant to the teaching evaluation changes to ensure their voices were included and to reduce additional resistance from their units. The committee also benefited extensively from the support of the Provost's office, who helped them work within the system and provided credibility to their work.
OU focused on two projects as part of their LCC4 work: An assessment of the Course Reflection Survey (CRS) as a feedback tool and a pedagogical partnerships program called Developing Understanding and Engagement Together (DUET).
DUET was started in 2023 as a program from the OU Center for Faculty Excellence (CFE). DUET is a structured pedagogical partnership program that builds relationships between students and faculty so they see each other as whole people. Each faculty participant has a student partner who collaborates with them to build an effective and welcoming classroom environment. The programming offered by DUET helps the faculty-student partners work together productively. Participants have gained a number of benefits related to teaching feedback, one of the main concerns for the Teaching Evaluation Working Group. The innovation of DUET is that the program recruits multiple faculty members from the same department, so that the recommendations can be disseminated with greater emphasis across the unit.
Students who have gone through DUET have collaborated with TEWG to help encourage students to fill out the Course Reflection Surveys in their courses. Faculty also now receive feedback from students through DUET and are more receptive to receiving feedback. The DUET participants also collaborated with the CFE on peer observation practices and how peer observations often did not elicit useful feedback for faculty to improve their teaching. Their feedback was incorporated into an online peer observation training, thus providing a resource for departments wishing to include peer observation as part of their evaluation of teaching. The training moved toward an inquiry-based approach rather than a 'telling them how to teach' approach.
The work of the Teaching Evaluation Working Group at OU was enhanced by participating in LT3 and HHMI IE3 more broadly in a few ways. The HHMI IE3 grant itself provided much-needed financial resources and an organizational structure that enabled them to expand the core of their mission. The grant provided the startup and maintenance funding for the DUET program, so that participants receive stipends for their time. The community formed by the grant provided an avenue to communicate regularly with partnered institutions engaging in pedagogical partnerships, which led to new ideas and validation for OU's initiatives. LT3, in particular, was a structured community of practice that allowed the OU committee to learn more readily from institutions engaged in similar projects focused on teaching evaluation reform, particularly about how student evaluations could be used in a more meaningful, credible way. The multi-institutional nature of the work also strengthened OU's position when presenting recommendations to administration, as they could reference specific practices at other institutions as well as how other institutions were learning from the changes OU was making to their teaching evaluation process.
Key Influences
This project was significantly influenced by the experiences of colleagues in LCC4 institutions. The Course Reflection Survey benefited greatly in its early development by connecting with the University of Oregon, who provided OU with insights into implementation timelines and other practical considerations for institutional change in course evaluations. OU also advantaged by engaging in deeper conversations about course evaluations and approaches to institutional change with several institutions during the LT3 meetings. The DUET program adapted and built upon Bryn Mawr's established, research-based model for pedagogical partnerships between students and faculty.