Accuracy and Apparent Accuracy in Medical Testing

Stuart Boersma and Teri Willard, Central Washington University
Author Profile
This material was originally developed by the the National Numeracy Network
as part of its collaboration with the SERC Pedagogic Service.

Summary

Unreviewed Activity submitted in preparation for the NNN Writing with Numbers Workshop

Students investigate two quantitative issues in the field of medical testing. First, students use two way tables and information about a diagnostic test's sensitivity and specificity to investigate the probability of a patient receiving a false positive result. Second, students learn about the phenomenon of referral bias in medical testing. Students use proportional reasoning to observe the effect of doctor referral rates on the apparent sensitivity and specificity of a screening tool.


Learning Goals

1. Extracting quantitative information from a complex environment.
  • Organizing data in a two-way table.
2. Use proportional reasoning to investigate conditional probabilities.
  • Given that a patient receives a "positive" on a medical test for a specific condition, what is the probability that the patient does in fact have the condition?
3. Explore quantitative reasoning in a realistic medical setting filled with discipline specific terminology.
  • Screening Tool
  • True and false positive/negative
  • Specificity
  • Sensitivity

Context for Use

For use in a "liberal arts" mathematics course. The activity is self-directed but students could benefit from occasional teacher clarification. One or two classroom discussions on the main points of the activity are recommended. The project will probably take two or three class days followed by additional homework time to create well written solutions.

Description and Teaching Materials

The Student Activity: Activity (Microsoft Word 85kB May9 08)

Excel Tool: This spreadsheet allows the instructor (or student) to quickly perform some of the calculations found in the student activity. Excel Tool (Excel 82kB May9 08)

Instructor notes: This comprehensive document provides solutions for instructors as well as ideas for further exploration. Instructor Notes (Microsoft Word 161kB May9 08)

Teaching Notes and Tips

By design, this project requires students to spend a fair amount of time extracting and using quantitative information in a complex setting. Consequently this project may contain terminology that is new to both student and teacher alike. It may be necessary to warn students ahead of time so that they are prepared to tackle this challenging reading and writing assignment.

Consider introducing this activity with a brief skit (Microsoft Word 2007 (.docx) 10kB May16 08) to get students thinking about the possibilities of true and false positives.

If this is the first time students have worked with data in two-way tables, it is suggested that Part 1 of the activity be completed by the instructor as an example. Students could then complete Part 2 individually or in pairs/groups. As a class, go over Part 2 to make sure everyone has "mastered" two-way tables and the types of probabilities that are being asked for. Students can then go on to complete Part 3 in their groups or on their own outside of class. It is highly recommended that some class time be then devoted to a discussion on the summary questions which appear at the end of Part 3. At this point, one of the main student outcomes should be a demonstrated understanding of the importatnce of the prevalence of the disease in computing the liklihood of a false positive. Instructors may which to extend this project by having students conduct further experiments. For example, how do these false readings change if only sensitivity is changed? What if only specificity is changed? Using a spreadsheet like the attached Excel Tool (Excel 82kB May9 08) would be a quick way for students to investigate these different scenarios.

Part 4 allows students to investigate a less familiar phenomenon: referral bias. Classroom use has indicated that students find this section more difficult. Instructors should be forewarned that students will need extra quidance and motivation for this part of the activity. However, we feel that this section should not be omitted as it provides students with a unique example from the medical literature.

Having students simply work through the activity is not enough. The instructor needs to spend some time emphasizing the main points and reminding students of the importance of the calculations they have been performing.

Assessment

As mentioned above, instructors may wish to use Parts 1 and 2 to provide formative assessment to the students (Have they understood the terminology? Can they fill in two-way tables? Can they calculated the requisit probabilities?) to clear up any initial confusions. Student responses to Parts 3 and 4 can be used for a more formal summative assessment.

References and Resources

Please see the related Numeracy (more info) article for a more detailed description of recent student and teacher assessment of this activity [link to article coming soon].

Portions of the Excel tool was based on this Java applet