University of Oklahoma

Institution Type: Very High Research Activity
MSI: no

Policy Level: Institution

Policy Status New course survey instrument ratified, initial policy regarding teaching evaluation in process

Keywords: Sources of information, Self-reflection, Student Experience Survey, Professional development

Overview

At the beginning of this work, there was little institution-level guidance on how faculty teaching should be evaluated. It was left to the colleges and departments to determine the methods and metrics.

In 2019, the Office of the Provost and the Faculty Senate organized the Teaching Evaluation Working Group (TEWG) charged with assessing and improving course evaluations and the faculty teaching evaluation process. Toward that end, the group has developed new instruments, gathered feedback on revised processes and tools from various stakeholder groups, and piloted new instruments to gather student feedback in classes. The results of this work have been posted on the Provost's website as suggestions for colleges and departments to adopt.

Theme 1: Align policy change with what matters

At the University of Oklahoma, there has been no single method or policy for gathering student feedback and incorporating that feedback into the teaching evaluation systems. Each department and/or college decides what their process will be. Guidance from the Office of the Provost indicates that student evaluations should not be the only metric used to evaluate faculty teaching effectiveness, but, in practice, that guidance has not always been heeded.

In the mid-2010s, there was a national push to change the nature of student evaluations of teaching. At the University of Oklahoma, the Student Senate came forward with a request to overhaul the evaluation system, focusing mainly on the end-of-course survey. The students didn't feel that the questions being asked were the right ones and, further, they had the sense that nothing changed after they gave their feedback.

Taking advantage of that moment in time, the Faculty Senate reached out to the Office of the Provost in 2019 to establish a partnership that became the Teaching Evaluation Working Group (TEWG). The charge to the group was to understand the current system for gathering student feedback and to make policy recommendations to improve data accuracy, actionability, and with the goal of modifying the evaluation of teaching in a way that would foster continuous improvement of teaching at OU. Everyone agreed that this had the potential to benefit all OU students.

In 2022, the University was awarded a five-year, multi-million dollar ADVANCE grant from the National Science Foundation (OU-Elevate). The grant is focused on the creation of a new model for faculty evaluation in teaching, research/creative activity, and service, and the work on teaching evaluation is heavily informed by the work of TEWG. Reviewing and revising teaching evaluation policy is a key part of the work of the grant. This policy change effort has administrative support and aligns with the university's strategic plan, which includes an effort to "develop fair and rigorous metrics for evaluating teaching effectiveness, and reward and promote outstanding performance."

Theme 2: Be strategic about policy content

A primary goal of this reform effort was to gather student feedback on their experiences in their courses in a way that the feedback could be used by instructors to improve their teaching.  The literature is clear that traditional student evaluations don't accurately reflect the quality of teaching for all members of the professoriate and may be skewed based on student perceptions disconnected from the actual teaching. There is also substantial evidence that the results of such student evaluations do not correlate with actual good teaching practices. One outcome of the work of the TEWG was the creation of a new student survey instrument, the CRS, that focuses on the educational experience students have in each class. Students generally do not have the knowledge and experience to meaningfully critique a faculty member's pedagogy, but they can express what they experienced in a course.

The TEWG also investigated promising practices in teaching evaluation that would minimize or mitigate documented issues with traditional teaching evaluation surveys. They recommended that colleges and departments incorporate multiple sources of evidence in evaluations of teaching quality, including the appropriate use of student voice as well as the faculty perspective.

Theme 3: Make policy change someone's job

The Faculty Senate and Provost's Office collaborated to establish the TEWG which is organized as an ad hoc group rather than a standing committee cemented in official policy. The Provost's Office provided dedicated IT and programming support that was critical for developing and implementing the project. Faculty members were responsible for crafting the content of the survey and making recommendations for more holistic evaluation of teaching. The group met with stakeholders, drafted recommendations for incorporating multiple sources of information into faculty teaching, developed and pilot-tested drafts of a new student experience survey, and developed resources to support departments in adapting to the new student experience survey. Subsequent funding from an NSF ADVANCE grant supported wide-scale re-envisioning of faculty evaluation policies. This facilitated further integration of the working group's recommendations into annual evaluations across campus and provided resources to sustain efforts required for long-term change. The members of the TEWG were asked to take on this work as part of their service to the institution.

Theme 4: Approach policy change as a process

In the process of making policy changes, the TEWG kept a timeline of key events and steps along the way to establishing the new teaching evaluation policy. The timeline helped ensure continuity through leadership changes and was also useful for responding to naysayers who questioned aspects of the change process (e.g., that input was sought, that changes were announced).

The Course Reflection Survey (CRS) was created, piloted, and successfully implemented across the university as the official student end-of-course survey. It is in the process of being formally incorporated into the faculty policy handbook for the university, a step that will require approval by the Faculty Senate, Provost, President, and Board of Regents.

Continuing work for the TEWG includes updating materials on the OU website to reflect the new thinking about faculty evaluation embodied in the new feedback form. Since the new survey is also not envisioned to be a primarily quantitative measure of faculty teaching, additional supports and documentation are envisioned to help chairs and departments use the feedback to evaluate higher-context aspects of teaching effectiveness.

References