About the Project

The On the Cutting Edge Classroom Observation Project aims to provide a direct measurement of changes in teaching practice. This project emerged from efforts to assess changes in teaching resulting from geoscience professional development, with the goal of moving beyond faculty self-reporting. Our goal is to observe a sufficiently large number of geoscience courses around the U.S. to be able to characterize how we are teaching in a selection of different types of higher education institutions (community colleges, comprehensive public universities, Research 1 universities, private colleges). This will allow us to assess the validity of the results of the On the Cutting Edge geoscience faculty surveys describing teaching methods and to better evaluate the impact of the On the Cutting Edge project.

We assembled and trained a team of observers drawn from a range of institutional types and representing a range of specializations and geographic regions. This team agreed to apply a common instrument to observations of teaching in at least ten geosciences classes. We selected the Reformed Teaching Observation Protocol (RTOP, Lawson et al., 2002; Sawada et al., 2002) as it represented an accessible, valid and reliable instrument designed to evaluate the presence of "reformed teaching" in college science and mathematics classrooms.

The privacy of participants is important to us. The Carleton College Institution Review Board has ruled this research "EXEMPT" Carleton IRB approval letter (Acrobat (PDF) 77kB Jan21 11). The notes taken by the faculty observer and the RTOP score (or any video produced for this scoring) will only be shared with the RTOP research team and the faculty participant (after the first round of analysis). Presentation of the material will be confined to project work groups, published reports, and professional conferences, names will not be identified. Any electronic information will be kept on a secure server at Carleton College.

Project History and Plan

In February of 2011 a group of 12 faculty met at Carleton College to analyze samples of geoscience classroom instruction, become trained in the RTOP protocol, discuss results, and calibrate scoring to produce uniform results comparable to those obtained by other groups.

The research group collected observations (either by direct observation or by video) of 30 other geoscience faculty. This first round of data was analyzed to understand the current characteristics of teaching in introductory and upper division courses in a variety of settings and to gain experience with the use of the RTOP instrument in characterizing teaching. In order to connect observations to self-report descriptions of teaching practice collected in the On the Cutting Edge surveys of geoscience faculty, participants will be asked to complete several survey questions.

Additional faculty were added to the 'certified RTOP observers' team after completing virtual training sessions, led by the leadership team in years two (2012) and three (2013) of the project. These trained faculty are expected to observe at least ten geoscience classrooms to contribute to generating a data set (200 total RTOP scores between 2011 and 2014) that characterizes geoscience teaching. We intend to use these results to better understand the meaning of the self-report survey data collected from a larger sample of faculty. We also intend to determine if any characteristics of teaching practice are correlated with participation in the On the Cutting Edge program or use of its website. A face-to-face meeting with the leadership team and RTOP observers will facilitate data analysis for all of the RTOP scores and development of a dissemination plan (papers and website).

We anticipate that this project will have a strong professional development component because analysis of teaching using RTOP provides substantial insight into one's own teaching practice and how it may be improved.