Author Profile

Statistics in Physics Lab: Catastrophic Cancellation

John Walkup, California State University-Fresno

Location:

Abstract

Error analysis consumes much of the focus in introductory physics labs. Catastrophic cancellation is a spike in error that occurs when subtracting two measurements of roughly equal magnitude. Often termed loss of significance or subtractive cancellation, this effect can easily relegate experimental results to utter worthlessness no matter how precise the measurements. A lab activity that exposes the ill effects of catastrophic cancellation in experimentation was carried out by undergraduate students that employs the traditional elastic collision experiment performed in countless labs across the country. Traditionally, lab designers try to lower experimental error as much as possible for students to confirm conservation of momentum and kinetic energy. In this lab activity, however, the calculations performed by the students were purposely modified to generate ridiculous levels of error based on nothing more than the order in which experimental values were summed. Students learn that measurement is not the only source of error in an experiment; that is, the order in which mathematical operations are carried out not only introduces error into calculations, but that this error can completely obscure experimental results.

Student Goals

  1. Students understand that the manner in which they perform arithmetical operations can generate ridiculously high error in results.
  2. Students understand that lab experiments often must be modified to eliminate problems with catastrophic cancellation.
  3. Students appreciate the importance of catastrophic cancellation, not only in experiment but also in computer programming.

Research Goals

  1. Uncover the extent to which catastrophic cancellation can negate the results of experiment.
  2. Develop guidelines for lab designers to avoid catastrophic cancellation in elastic collision experiments.

Context

Twelve students in one introductory physics lab completed a lab experiment specifically designed by the advisor to generate significant catastrophic cancellation. The traditional elastic collision lab involving an air track, motion sensors, and moveable gliders served as the vehicle for the experiment. Over the summer, three students performed more refined trials and helped prepare an article for submission to Physics Education. (Update: The article has been published.)

Target Audience:
CURE Duration:

CURE Design

Twelve students worked in teams of three using the traditional elastic collision lab experiment. The lab procedure was purposely designed to provide good results and bad results from the same research data; the only difference being the order in which arithmetic operations were conducted. Students brainstormed on the causes of such a contradiction. The advisor then instructed them to investigate the source of the problem by focusing them on the error generated when subtracting two numbers of roughly equal magnitude. Three students then performed refined trials over the summer of 2019 for publication in a refereed journal.

The elastic collision lab is a mainstay of physics instruction. However, inattention to the order in which mathematical operations are carried out drives statistical error up to the point where results are worthless. Lab designers across the country can learn from this research how to avoid this problem. The students' work has since been published in the peer-reviewed academic journal Physics Education.

Core Competencies:
Nature of Research:

Tasks that Align Student and Research Goals

Research Goals →
Student Goals ↓
Research Goal 1: Uncover the extent to which catastrophic cancellation can negate the results of experiment.
Research Goal 2: Develop guidelines for lab designers to avoid catastrophic cancellation in elastic collision experiments.

Student Goal 1: Students understand that the manner in which they perform arithmetical operations can generate ridiculously high error in results.

Students collide two gliders of equal mass and speed in an elastic collision on an air track. Students use a motion sensor and software to capture the speed of each cart before and after the collision. Students then calculate the total momenta and energy of each glider before and after the collision.

Students brainstorm to understand why the results between Student Goal 2 varies so markedly from results of Student Goal 3.


Student Goal 2: Students understand that lab experiments often must be modified to eliminate problems with catastrophic cancellation.

Students compare the total momentum and energy of both gliders combined before and after the collision and compute the relative difference for both physical properties (ideally 0 for elastic collisions).

Students reexamine their results in the light of catastrophic cancellation, which the advisor has asked them to investigate.


Student Goal 3: Students appreciate the importance of catastrophic cancellation, not only in experiment but also in computer programming.

Students compare the loss of momentum and energy of one glider to the gain in momentum and energy of the second glider (ideally equivalent for elastic collisions).

Students formulate new lab procedures that instruct future lab designers on how to structure the elastic collision lab to avoid catastrophic cancellation.


Instructional Materials

Lab Manual Handout (Microsoft Word bytes Jan27 20)

Instructional Staffing

The only support staff was the lab instructor. The physics department's lab director helped ensure that the proper equipment was available.

Author Experience

John Walkup, California State University-Fresno

This CURE project centered on data collected by students in an introductory physics lab. We refined the existing lab manual to investigate an important issue in statistical analysis. We think that this project will help other labs refine their own lab procedures.

Advice for Implementation

If instructors choose to deploy this activity as a guided inquiry activity, they should avoid informing the students of catastrophic cancellation. Instead, students should brainstorm why their results were so satisfying with one approach and so dissatisfying with the other. Sharper students could reason that subtracting numbers that are close in value had something to do with the inconsistency.

There is little need to use top-quality equipment for this lab, since this lab is supposed to use error (such as friction) as a teaching tool.

Resources

The article published as a result of this CURE, as well as the references contained therein, are probably all that is needed. https://iopscience.iop.org/article/10.1088/1361-6552/ab51fb