Using Learning Assistants in Oral Assessments

To increase student learning

Oral assessments have been shown to improve student conceptual understanding, student grades, and increase the number of students passing the course at the University of Colorado (see Assessment, below). Prior research has also shown that both testing and feedback are crucial to improving learning (Karpicke and Roediger, 2008; Nicole & Macfarlane-Dick, 2006; Gibbs & Simpson, 2004).

To increase student engagement

Oral assessments are by nature student-centered, active-engagement activities. Additionally, oral assessments have improved course attendance. During the assessment students learn to help each other by asking questions and other active techniques rather than working problems for one another.

To emphasize sensemaking and reasoning

Oral assessments emphasize the reasoning behind answers, providing individualized feedback on student thinking. A major goal of the orals is to help students articulate their thinking, make sense of ideas and make mathematical connections.

To learn more about student thinking

By using oral assessments, instructors, Teaching Assistants and Learning Assistants learn more about how students are thinking about the material and see firsthand the power of active learning.

Any undergraduate course could make use of oral assessments (assuming that appropriate materials exist or can be developed). At the University of Colorado, the introductory Calculus I and Calculus II courses have developed and used oral assessments regularly. Other courses (Calculus III, Matrix Algebra, Mechanical Engineering's Statics and Component Design, and an Aerospace Engineering course) have also begun to use or develop oral assessments.

Students from the course sign up to attend one or more oral assessments in the days preceding an exam. Assessments are attended by small groups of 5-6 students, who stand at individual blackboard or whiteboard spaces and answer questions posed by a facilitator using sketches or explanations. They may work together to help each other answer the question, and the facilitator asks prompting questions to help them think about the material. The assessments are ungraded, meant primarily to help students gain a deeper understanding of the material. Oral assessments are a voluntary activity and do not contribute to students' grades.

What are oral assessments?

Oral assessments:

• Are voluntary and ungraded
• Are one hour long
• Are offered during the two days prior to the exam
• Are attended by groups of 5-6 students at a time in a room with chalk- or whiteboard space.
• Students may attend more than one oral assessment per exam (some attend 3 or 4).
• Facilitated by instructors, Teaching Assistants and Learning Assistants
• Consist of scripted conceptual questions
• Require students to help one another first, with the facilitator present to help them overcome hurdles that they aren't getting past on their own.

What is the process of running an oral assessment?

The orals process (as described by the CU-Boulder Applied Math Department) is as follows:

1. Students write their name on their whiteboard or chalkboard space
2. Facilitator asks the first question and directs students to write the question on their board.
3. Students are directed to think about the idea and to feel free to "draw pictures, talk about the problem with others, or write anything on the board that you think will help you move forward"
4. As students work, facilitator encourage students to discuss it amongst themselves, and re-poses student questions to the group as a whole. Students are asked to check with one another to ensure that they all truly understand the main ideas.

What kinds of questions are asked in an oral assessment?

Oral assessment questions are focused on mathematical connections and meaning, such as asking students to graph, draw, and explain. They do not ask a student, for example, to find the derivative of a function.

Some example questions from orals:

• Explain why you would use linearization
• What is the purpose of linearization
• Draw a representation of the linearization of y=sin(x) at x=0
• Explain the relationship between the formula for linearization and the graph that you just drew.

More mathematics example oral questions with embedded facilitator instructions.

How are Learning Assistants used in oral assessments?

Learning Assistants have been instrumental in both:

• Facilitating oral assessments
• Developing scripted questions for oral assessments

Unlike instructors and Teaching Assistants, Learning Assistants have a fresh memory of what it was like to learn this material. Along with other students in the group, Learning Assistants can share how they made sense of the material. Learning Assistants have been prepared to ask effective guiding questions.

How are oral assessments different from a review session?

Orals provide a much more intimate environment for students to interact with instructors and Learning Assistants and deeply learn the material. Unlike a review session, which is generally taught with an instructor at the blackboard and students following along, oral assessments require students to be actively engaged in answering questions and justifying their reasoning.

Prepare facilitators (including Learning Assistants)

• At CU-Boulder, Learning Assistants assisting with oral assessments generally have already participated in oral assessments themselves.
• Facilitators receive a 2-hour training in the first two weeks of class, including the research behind the orals, assessment data on their effectiveness, a mock oral using student volunteers, and a discussion/debriefing about the mock oral. Check the CU-Boulder Applied Mathematics oral assessments website for a training video and facilitator's guide to be posted.
• Prior to the oral, facilitators receive two copies of the scripted questions: One copy that has only the scripted questions, and another with the scripted questions plus additional prompts to ask in the face of particular student difficulties.

Provide tips on facilitating student discussion

A pedagogy course will prepare Learning Assistants to facilitate student interaction, but they will need some tips on the promoting student discussion during oral assessments. Tips on facilitation include:

• Ask students to draw pictures and graphs
• Relate the questions to prior knowledge and concepts
• Ask questions that probe deep understanding.
  Show probing question examples
  Hide
  "Why is that?" - "What does this have to do with what you said earlier?" - "What does this have to do with what we learned in class?" - "What does that remind you of?" - "Can you think of another way of saying that?" - "If you were trying to explain this someone who didn't understand, what would you say?" - "Why can't you just ______ here?" - "How are _____ and ____ related?"
• Circulate the room to inspect student work
• If one student is still struggling with an idea but the others are finished, give them a new problem while you work with the struggling student.
• Ask questions to help students understand, but don't dwell too long on a single problem. If a student is really stuck, help them through a step and then ask them to do it on their own on the next problem.
• If one student is doing all the talking, ask him to help other students who are still struggling, or to give them a chance to speak
• If one student doesn't talk much, direct questions to them directly (having students' names on the board makes this easier).

See Heller and Hollabaugh (1992) for more information on managing productive groups, and see also the Tips & Strategies sheet (Acrobat (PDF) 107kB Jul27 10) for a document with ideas on probing student thinking.

Recruit students to attend the orals

Tell students the dates that oral assessments will be available, and remind them several times to sign up. Encourage them to attend, indicating the data suggesting that this can help them on the exams. Provide an easy sign-up process. Students should be encouraged to prepare for the oral examination by studying ahead of time.

Tell students what to expect

In their undergraduate career, students are rarely asked to explain their reasoning or to work in groups. Thus, it is often necessarily to explain to students what will be expected of them in the activity, and to repeat that explanation often. This will both help the students to work productively and make the Learning Assistants' job easier.

Arrange for a space for the orals

Find a quiet room with ample chalkboard and/or whiteboard space for each student and for the group as a whole.

Meet weekly with Learning Assistants to discuss the class

Weekly meetings also allow Learning Assistants a chance to describe challenges that they've faced so that you can give them tips and strategies, as well as a supportive environment, to provide guidance in their development as teachers. This is also a time for you to collect their observations – either verbally or through written field notes – and get feedback on what they have observed in the class. Find more information about How to Teach with Learning Assistants.

• Oral Assessments can take time to create, and facilitator training also takes time
• Orals have logistical challenges, such as arranging for room space and student and facilitator sign-ups
• Creating a supportive learning environment, where students buy in to the activity, can be a challenge.

Oral assessments have had positive impacts on student learning in Calculus I and II, even taking motivation and preparation into account:

• Student exam grades were higher for students who attended orals in Calculus I than those who did not. This was true for students at varying levels of preparation.
• The course failure rate decreased significantly in both Calculus I and II with the implementation of orals
• Student comments indicated that they saw benefit to the orals. For example, on student commented that orals "helped me understand the hard concepts" or "helps by allowing me to hear how other students think about some of these things."

This same data could be collected in your own courses to assess the impact of orals.

Oral assessments page for CU-Boulder Applied Math: Includes sample questions, tips for facilitation, and guidelines for students.

Gibbs, G and Simpson, C. (2004). "Conditions under which assessment supports students' learning." Learning and Teaching in Higher Education vol.1 pp.3-31.

Heller, P. and Hollabaugh, M. (1992). "Teaching problem solving through cooperative grouping. Part 2: Designing problems and structuring groups", Am. J. Phys, 60, 637-644. A good reference on structuring and managing cooperative groups.

Karpicke, J.D. and Roediger, H.L. (2008). "The critical importance of retrieval for learning," Science, 319, 966-968.

Nicol, D, J. & Macfarlane-Dick (2006), "Formative assessment and self-regulated learning: A model and seven principles of good feedback practice," Studies in Higher Education,31(2), 199-218.