Interpreting your RTOP Score

Components of the RTOP Scale

The Reformed Teaching Observation Protocol (RTOP; Lawson et al., 2002, Sawada et al., 2002) provides a standardized means for detecting the degree to which classroom instruction uses student-centered, engaged learning practices. It was designed to measure the quality of instruction in college science and mathematics courses.

The RTOP protocol uses five subscales with five items on each subscale with each item described on a five point (0-4) Likert scale. (Text in parentheses is clarifying text used by project participants to talk about the Subscales in the context of this project.) To get a general idea for how your teaching falls on a scale of traditional lecture to reformed teaching, see these Vignettes.

Subscale 1 - Lesson Design and Implementation: What the teacher intended to do

This subscale examines the design and application of a lesson to determine if it is sufficient to support student understanding. Items examine how the instructor organizes the lesson to honor students' preconceptions from other classes and everyday experiences, and to provide opportunities to explore aspects of the topic prior to formal instruction. Research indicates that students enter the classroom with many preconceptions and in order to overcome those misconceptions, an instructor must learn from the students what ideas they bring to the class so that they can be addressed through interventions (NRC, 2000).

Other items in this subscale recognize the role of the social construction of knowledge and seek evidence that the instructor provided opportunities for students to work together in groups. Learning communities in which students feel they can actively be a part of the learning experience create positive learning environments which are critical for motivating student learning (Weinstein et al., 2006). 

At their most successful, student activities will require divergent thinking and support more than one approach to problem solving. An ideal lesson would have sufficient flexibility to accommodate student input and questions that may redirect the path of the lesson. An inquiry-based approach to teaching has been documented as a more authentic way of learning science and leads to greater learning gains. Inquiry can simply be described as exploration preceding explanation (NRC, 1996), and should be done in meaningful ways in which students are able to work through realistic scientific problems to which there are more than one correct answer (not just a "hands-on" activity; Bybee, 2000).

Learn tips on how to improve Subscale 1: Lesson Design and Implementation scores.

Subscale 2 - Propositional Pedagogic Knowledge: What the teacher knows, and how well they are able to organize and present material in a learner-oriented setting

This scale addresses what the character of the content the instructor teaches and their command of the material. For example, given the context of the course, does the lesson highlight fundamental concepts and are these concepts presented clearly to illustrate the relationships among key components. Research indicates that experts see content in a framework with big conceptual ideas supporting details; novices tend to view the supporting details and lack the conceptual framework (Chi, 1981). It is important that as experts we present our content in a clear framework upon which our students can pin the details (NRC, 2000).

Items in this subscale assess the instructor's knowledge of the material under discussion. In addition, some items explore how the lesson incorporates ways for students to represent (e.g., graphs, equations) abstract concepts that are central to developing thorough conceptual knowledge. In geology we deal with concepts that are not directly observable, using representations that support student development of these abstract concepts becomes critical for successful student learning of the content (Jee et al., 2010).

Many students feel a disconnect between what they learn in class and the relevance to their everyday lives or other science classes they've taken. This subscale seeks to identify how new knowledge is integrated with other disciplines and real world applications. Supporting students by helping them to make these connections will generate more positive learning experiences (NCTM,2000).

Learn tips on how to improve Subscale 2: Propositional Knowledge scores.

Subscale 3 - Procedural Pedagogic Knowledge: What the students did

This scale addresses what the students are asked to do within the classroom and includes a variety of higher order skills, tools and strategies an instructor might choose to employ in order to support student learning of the content. Much of this subscale examines the scientific ways of knowing and if students are engaged in this process in the classroom. Scientific ways of thinking (predicting, estimating, hypothesizing, negotiating ideas, and alternative ways of reasoning) are vital for our students to understand what science is and to develop critical thinking skills. The more opportunities that students have to model authentic scientific tasks, the more likely they are to start thinking scientifically (Abd-El Khalick & Lederman, 2000).

This subscale focuses on thought-provoking activities that involve both the synthesis of concepts and the assessment of procedures necessary to solve problems. Instructor assigned tasks should have the potential to prompt the evaluation of ideas and the negotiation of meaning through constructive dialogue or debate. Not only should students be thinking deeply about the content, but they should also be considering their own learning and reflecting upon both what they learned and how they learned it. Research indicates that student self-regulation of learning is critical for student success and persistence (Zimmerman, 2001). Self-regulation of learning includes thinking about where one is in their process of learning, and what would be next steps to a given procedure. In addition, metacognition is a critical component. Metacognition measures how students think about their learning, and recognize their own strengths and weaknesses. The more time that students have to reflect on these learning opportunities, the more student learning will occur (NRC, 2000).

Learn tips on how to improve Subscale 3: Procedural Knowledge scores.

Subscale 4 - Student-Student Interaction

This scale evaluates the number and type of interactions among students and how the instructor facilitates such interactions. In reformed classes, students are not passive listeners, but rather actively communicating with one another. This process of explaining their own ideas and evaluating the ideas of others is key to the development of student critical thinking skills.

Research shows that student-student interactions have many positive effects on learning including increased understanding of key concepts (Smith et al., 2009) and increased problem-solving ability (Hake, 1997). Chi (2009) defines a spectrum of student actions that range from active (e.g., paraphrasing text, manipulating objects) to constructive (e.g., explaining, reflecting, planning) to interactive (e.g., building on partners contribution, debating). While all of these may be considered active learning under reformed instruction, only the latter activities expressly involve student-student interaction. 

Student-student interactions help students to assess their understanding of course concepts, help to develop teamwork and effective communication skills, and allow students to discuss their questions and ideas in a low-stakes environment. Under ideal conditions, all students would have an opportunity to have their ideas heard and respected. This may occur in small group settings or in larger class-wide discussions.

Learn tips on how to improve Subscale 4: Student-Student Interaction scores.

 

Subscale 5 - Student-Instructor Interaction

This scale addresses the culture of respect and comfort in the classroom as supported by both learners and teacher. In a reformed classroom, the teacher fosters a culture where students feel comfortable asking questions and have control over their own learning process. A teacher in this classroom demonstrates patience, listens to students, and acts as a resource for student learning. As class size increases it becomes more challenging for the instructor to accurately assess the depth of learning and identify potential misconceptions harbored by students.

Research indicates that students learn best when they actively engage in learning (Hake, 1998), but they need to feel safe and comfortable in taking risks to do so (Pekrun, 2002). A teacher who supports students in this experience creates learning environments where students are able to take those risks. Providing students the opportunity to have executive control over their learning process both empowers them to learn and increases their overall learning gains (Weinstein et al., 2006).

Teachers support student learning when they listen carefully to student questions, this means beyond simply answering a question, but helping students to make connections between other questions and the content (Ruiz-Primo & Furtak, 2007). In addition, providing the "wait time" needed for all students to effectively think through problems or questions is critical for student success (NCTM, 2000).

Learn tips on how to improve Subscale 5: Student-Teacher Interaction scores.