Classroom Observation Protocols

Introduction

Observation protocols provide a means to collect standardized data about teaching practices.

Developing and testing our observation protocols

We chose to implement existing two validated and reliable observation protocols in the TIDeS project in order to provide opportunities for comparison with a larger dataset. The two instruments we are using in this project are:

• The Classroom Observation Protocol for Undergraduate STEM (COPUS) (Smith et al., 2013), with the addition of the formative assessment codes identified by McConnell et al. (2021);
• The Science Discourse Instrument (SDI) (Fishman et al., 2017).

COPUS allows the observer to record the activities of students and instructors in 2-minute increments, providing an overview of student and instructor practices and students' level of engagement. While COPUS can reliably characterize how instructors and students are spending their time in the classroom, it does not assess the quality or context of the classroom practices captured. To capture the quality of interactions and discourse, we added the coding for formative assessments proposed by McConnell et al. (2021) and used the COPUS to identify 15-minute segments of each class to apply the SDI, which is used to identify and analyze discussion features including purpose, structure, references to discussion norms, participation, linkages across relevant contexts, and think time.

Because these are well-established instruments, our testing focused on establishing inter-rater reliability and best practices for making observations across the project. To do so, we observed and recorded on video three different undergraduate introductory science classrooms across different modalities: online real-time (via Zoom), in-person, and hybrid. The video recordings allowed us to iteratively test our COPUS codes until we came to consensus and to select 15-minute segments for coding with the SDI. Selected 15-minute segments were transcribed and coded by two researchers until agreement was reached.

Implementing the observation protocols

We are observing TIDeS materials developers in their classes prior to implementing the new materials, and we will observe them again when they implement the new materials in order to assess the impacts on their teaching practices and classroom interactions. For each instructor, we observe and record two to four class meetings, using the COPUS instrument in real-time and applying the formative codes and conducting SDI analysis from the recording. 

Current Versions of the Instruments

Adapted COPUS full Snapshot (Acrobat (PDF) 121kB Oct26 21)
Adapted COPUS protocol (Acrobat (PDF) 123kB Oct26 21)
Adapted SDI Snapshot (Acrobat (PDF) 63kB Oct26 21)
Observation Protocol Spreadsheet (Excel 2007 (.xlsx) 320kB Dec16 21)

References

Fishman, E. J., Borko, H., Osborne, J., Gomez, F., Rafanelli, S., Reigh, E., ... & Berson, E. (2017). A practice-based professional development program to support scientific argumentation from evidence in the elementary classroom.Journal of science teacher education, 28(3), 222-249.

McConnell, M., Boyer, J., Montplaisir, L. M., Arneson, J. B., Harding, R. L., Farlow, B., & Offerdahl, E. G. (2021). Interpret with Caution: COPUS Instructional Styles May Not Differ in Terms of Practices That Support Student Learning. CBE—Life Sciences Education, 20(2), ar26.

Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE—Life Sciences Education, 12(4), 618-627.