Interpreting your RTOP Score
Components of the RTOP Scale
The RTOP protocol uses five subscales with five items on each subscale with each item described on a five point (0-4) Likert scale. (Text in parentheses is clarifying text used by project participants to talk about the Subscales in the context of this project.)
Subscale 1 - Lesson Design and Implementation: What the teacher intended to do
Subscale 1 items:
- Instructional strategies and activities respected students' prior knowledge and the preconceptions inherent therein.
- The lesson was designed to engage students as members of a learning community.
- In this lesson, student exploration preceded formal presentation.
- This lesson encouraged students to seek and value alternative modes of investigation or of problem solving.
- The focus and direction of the lesson was often determined by ideas originating with students.
This subscale examines the design and application of a lesson to determine if it is sufficient to support student understanding. Items examine how the instructor organizes the lesson to honor students' preconceptions from other classes and everyday experiences, and to provide opportunities to explore aspects of the topic prior to formal instruction. Research indicates that students enter the classroom with many preconceptions and in order to overcome those misconceptions, an instructor must learn from the students what ideas they bring to the class so that they can be addressed through interventions (NRC, 2000).
Other items in this subscale recognize the role of the social construction of knowledge and seek evidence that the instructor provided opportunities for students to work together in groups. Learning communities in which students feel they can actively be a part of the learning experience create positive learning environments which are critical for motivating student learning (Weinstein et al., 2006).
At their most successful, student activities will require divergent thinking and support more than one approach to problem solving. An ideal lesson would have sufficient flexibility to accommodate student input and questions that may redirect the path of the lesson. An inquiry-based approach to teaching has been documented as a more authentic way of learning science and leads to greater learning gains. Inquiry can simply be described as exploration preceding explanation (NRC, 1996), and should be done in meaningful ways in which students are able to work through realistic scientific problems to which there are more than one correct answer (not just a "hands-on" activity; Bybee, 2000).
Subscale 2 - Propositional Pedagogic Knowledge: What the teacher knows, and how well they are able to organize and present material in a learner-oriented setting
Subscale 2 items:
- The lesson involved fundamental concepts of the subject.
- The lesson promoted strongly coherent conceptual understanding.
- The teacher had a solid grasp of the subject matter content inherent in the lesson.
- Elements of abstraction (i.e., symbolic representations, theory building) were encouraged when it was important to do so.
- Connections with other content disciplines and/or real world phenomena were explored and valued.
This scale addresses what the character of the content the instructor teaches and their command of the material. For example, given the context of the course, does the lesson highlight fundamental concepts and are these concepts presented clearly to illustrate the relationships among key components. Research indicates that experts see content in a framework with big conceptual ideas supporting details; novices tend to view the supporting details and lack the conceptual framework (Chi, 1981). It is important that as experts we present our content in a clear framework upon which our students can pin the details (NRC, 2000).
Items in this subscale assess the instructor's knowledge of the material under discussion. In addition, some items explore how the lesson incorporates ways for students to represent (e.g., graphs, equations) abstract concepts that are central to developing thorough conceptual knowledge. In geology we deal with concepts that are not directly observable, using representations that support student development of these abstract concepts becomes critical for successful student learning of the content (Jee et al., 2010).
Many students feel a disconnect between what they learn in class and the relevance to their everyday lives or other science classes they've taken. This subscale seeks to identify how new knowledge is integrated with other disciplines and real world applications. Supporting students by helping them to make these connections will generate more positive learning experiences (NCTM,2000).
Subscale 3 - Procedural Pedagogic Knowledge: What the students did
Subscale 3 items:
- Students used a variety of means (models, drawings, graphs, symbols, concrete materials, manipulatives, etc.) to represent phenomena.
- Students made predictions, estimations, and/or hypotheses (PEH) and devised means for testing them.
- Students were actively engaged in thought-provoking activity that often involved the critical assessment of procedures.
- Students were reflective about their learning.
- Intellectual rigor, constructive criticism, and the challenging of ideas were valued.
This scale addresses what the students are asked to do within the classroom and includes a variety of higher order skills, tools and strategies an instructor might choose to employ in order to support student learning of the content. Much of this subscale examines the scientific ways of knowing and if students are engaged in this process in the classroom. Scientific ways of thinking (predicting, estimating, hypothesizing, negotiating ideas, and alternative ways of reasoning) are vital for our students to understand what science is and to develop critical thinking skills. The more opportunities that students have to model authentic scientific tasks, the more likely they are to start thinking scientifically (Abd-El Khalick & Lederman, 2000).
This subscale focuses on thought-provoking activities that involve both the synthesis of concepts and the assessment of procedures necessary to solve problems. Instructor assigned tasks should have the potential to prompt the evaluation of ideas and the negotiation of meaning through constructive dialogue or debate. Not only should students be thinking deeply about the content, but they should also be considering their own learning and reflecting upon both what they learned and how they learned it. Research indicates that student self-regulation of learning is critical for student success and persistence (Zimmerman, 2001). Self-regulation of learning includes thinking about where one is in their process of learning, and what would be next steps to a given procedure. In addition, metacognition is a critical component. Metacognition measures how students think about their learning, and recognize their own strengths and weaknesses. The more time that students have to reflect on these learning opportunities, the more student learning will occur (NRC, 2000).
Subscale 4 - Student-Student Interaction
Subscale 4 items:
- Students were involved in the communication of their ideas to others using a variety of means and media.
- The teacher's questions triggered divergent modes of thinking.
- There was a high proportion of student talk and a significant amount of it occurred between and among students.
- Student questions and comments often determined the focus and direction of classroom discourse.
- There was a climate of respect for what others had to say.
This scale evaluates the number and type of interactions among students and how the instructor facilitates such interactions. In reformed classes, students are not passive listeners, but rather actively communicating with one another. This process of explaining their own ideas and evaluating the ideas of others is key to the development of student critical thinking skills.
Research shows that student-student interactions have many positive effects on learning including increased understanding of key concepts (Smith et al., 2009) and increased problem-solving ability (Hake, 1997). Chi (2009) defines a spectrum of student actions that range from active (e.g., paraphrasing text, manipulating objects) to constructive (e.g., explaining, reflecting, planning) to interactive (e.g., building on partners contribution, debating). While all of these may be considered active learning under reformed instruction, only the latter activities expressly involve student-student interaction.
Student-student interactions help students to assess their understanding of course concepts, help to develop teamwork and effective communication skills, and allow students to discuss their questions and ideas in a low-stakes environment. Under ideal conditions, all students would have an opportunity to have their ideas heard and respected. This may occur in small group settings or in larger class-wide discussions.
Subscale 5 - Student-Instructor Interaction
Subscale 5 items:
- Active participation of students was encouraged and valued.
- Students were encouraged to generate conjectures, (or) alternative solutions, and/or different ways of interpreting evidence.
- In general the teacher was patient with the students.
- The teacher acted as a resource person, working to support and enhance student investigations.
- The metaphor "teacher as listener" was very characteristic of this classroom.
This scale addresses the culture of respect and comfort in the classroom as supported by both learners and teacher. In a reformed classroom, the teacher fosters a culture where students feel comfortable asking questions and have control over their own learning process. A teacher in this classroom demonstrates patience, listens to students, and acts as a resource for student learning. As class size increases it becomes more challenging for the instructor to accurately assess the depth of learning and identify potential misconceptions harbored by students.
Research indicates that students learn best when they actively engage in learning (Hake, 1998), but they need to feel safe and comfortable in taking risks to do so (Pekrun, 2002). A teacher who supports students in this experience creates learning environments where students are able to take those risks. Providing students the opportunity to have executive control over their learning process both empowers them to learn and increases their overall learning gains (Weinstein et al., 2006).
Teachers support student learning when they listen carefully to student questions, this means beyond simply answering a question, but helping students to make connections between other questions and the content (Ruiz-Primo & Furtak, 2007). In addition, providing the "wait time" needed for all students to effectively think through problems or questions is critical for student success (NCTM, 2000).