Accuracy and Apparent Accuracy in Medical Testing
as part of its collaboration with the SERC Pedagogic Service.
Students investigate two quantitative issues in the field of medical testing. First, students use two way tables and information about a diagnostic test's sensitivity and specificity to investigate the probability of a patient receiving a false positive result. Second, students learn about the phenomenon of referral bias in medical testing. Students use proportional reasoning to observe the effect of doctor referral rates on the apparent sensitivity and specificity of a screening tool.
- Organizing data in a two-way table.
- Given that a patient receives a "positive" on a medical test for a specific condition, what is the probability that the patient does in fact have the condition?
- Screening Tool
- True and false positive/negative
Context for Use
Description and Teaching Materials
The Student Activity: Activity (Microsoft Word 85kB May9 08)
Excel Tool: This spreadsheet allows the instructor (or student) to quickly perform some of the calculations found in the student activity. Excel Tool (Excel 82kB May9 08)
Instructor notes: This comprehensive document provides solutions for instructors as well as ideas for further exploration. Instructor Notes (Microsoft Word 161kB May9 08)
Teaching Notes and Tips
By design, this project requires students to spend a fair amount of time extracting and using quantitative information in a complex setting. Consequently this project may contain terminology that is new to both student and teacher alike. It may be necessary to warn students ahead of time so that they are prepared to tackle this challenging reading and writing assignment.
Consider introducing this activity with a brief skit (Microsoft Word 2007 (.docx) 10kB May16 08) to get students thinking about the possibilities of true and false positives.
If this is the first time students have worked with data in two-way tables, it is suggested that Part 1 of the activity be completed by the instructor as an example. Students could then complete Part 2 individually or in pairs/groups. As a class, go over Part 2 to make sure everyone has "mastered" two-way tables and the types of probabilities that are being asked for. Students can then go on to complete Part 3 in their groups or on their own outside of class. It is highly recommended that some class time be then devoted to a discussion on the summary questions which appear at the end of Part 3. At this point, one of the main student outcomes should be a demonstrated understanding of the importatnce of the prevalence of the disease in computing the liklihood of a false positive. Instructors may which to extend this project by having students conduct further experiments. For example, how do these false readings change if only sensitivity is changed? What if only specificity is changed? Using a spreadsheet like the attached Excel Tool (Excel 82kB May9 08) would be a quick way for students to investigate these different scenarios.
Part 4 allows students to investigate a less familiar phenomenon: referral bias. Classroom use has indicated that students find this section more difficult. Instructors should be forewarned that students will need extra quidance and motivation for this part of the activity. However, we feel that this section should not be omitted as it provides students with a unique example from the medical literature.
Having students simply work through the activity is not enough. The instructor needs to spend some time emphasizing the main points and reminding students of the importance of the calculations they have been performing.
References and Resources
Portions of the Excel tool was based on this Java applet