LT3: Sources of Evidence
How do we assess progress in evaluation of inclusive teaching using a variety of evidence-based indicators and measures, with the aim of improving evaluation tools and processes over time?
We propose that our institutions do not have shared expectations, tools, and measures for evaluating inclusive excellence in teaching at the unit level (departments and divisions). In addition, the relationship between inclusive excellence as an aspiration and the principles of anti-racist, equity pedagogies are often fraught with tension. The two approaches place differing emphasis on the necessity of attending to the root causes of exclusionary practices. Our sense is that most institutions in our LCC have not yet determined data sources or data use practices that would be most meaningful to evaluate and catalyze action toward widespread inclusive teaching and equity pedagogies. Thus, our final project aims to gather assessment tools such that instructors and departments or other units may act with collective agency to unlearn exclusive, hierarchical, racist practices and to experiment with, assess, learn, and refine inclusive, equitable, anti-exploitative, and anti-racist pedagogies and curricula. This project will focus on developing annotated cards or summary sheets that provide brief descriptions of effective and promising tools, the actual tool itself, key references, and recommended approaches to ensure robust and holistic evaluation of inclusive teaching.
Through this project, we expect that all of our institutions will develop expectations and measures to be used in the evaluation of equity pedagogies and inclusive teaching. Additionally, instructors will not feel threatened by the resultant evaluation. The goals and specific metrics for success will include:
- An entire campus community (instructors, students, administrators) recognizes the
necessity of using multiple forms of evidence to assess teaching—both formatively,
developmentally, and summatively—that includes demonstration of student learning. - Stakeholders have insights into equitable learning based on student grades and
opportunity data available to all instructors/departments. - Institutions have clear expectations and valid measures (e.g., disparities, belongingness)
for evaluating whether teaching is inclusive (process and outcomes), both within and
across institutions, and make these data available, through a regular cycle of
assessment. - Evaluators have tools to measure faculty progress towards inclusive teaching practices.
Our process for carrying out this project will involve the following steps:
- Examine specific, evidence-based ways to determine the extent to which inclusive teaching is happening currently across the LCC4 institutions and beyond by (a) identifying suitable frameworks for documenting inclusive teaching (e.g., "the three voice" framework of student, peer, and self evaluation); (b) determining currently used approaches and potential additions or refinements to teaching about racism, sexism, classism, ableism, and other forms of oppression in scientific disciplines (including the historical and contemporary contributions of the disciplines to systems of oppression), and how these vary in institutional settings with varying histories, traditions, and representational diversity of the student body, faculty and staff; (c) compiling and preparing cards/summaries describing the different tools and approaches that are potentially useful for measuring inclusive teaching and equity pedagogies from multiple perspectives (e.g., observation protocols, peer evaluation training materials, self-reflection templates, student experience surveys, retention and grade data); (d) offering recommendations about how such tools can be used within the low-stakes diagnostic/formative settings vs. high-stakes evaluative settings; and (e) assessing gaps in available tools and approaches and develop tools and approaches to fill those gaps.
- Pilot the most promising of the identified tools and approaches, including (a) identifying pros and cons and refining based on results of pilot testing of diagnostic/formative, low-stakes teaching evaluation during instructor development to inform continuous improvement (intersects with Project 2) and summative, high-stakes teaching evaluation such during merit review, appointments, promotion, and tenure (intersects with Project 1); and (b) where possible, offering guidance on use of teaching evaluation measures and processes to to assess sustainability of inclusive teaching longitudinally.
- Evaluate impact, reflect, and share lessons learned by (a) sharing progress and getting feedback on the translations, including obstacles encountered (e.g., which type of faculty do/do not make progress toward inclusive teaching) and ideas about how they can be mitigated or overcome; and (b) evaluating the extent to which translations were successful, including documenting obstacles faced during the translation and whether and how these obstacles were navigated.