Statistics and Error Rates in Death Penalty Cases

This page authored by Bill Peterson, based on a report in CHANCE News 9.07

Author Profile
This material was originally developed through CAUSE
as part of its collaboration with the SERC Pedagogic Service.

Initial Publication Date: May 17, 2007

Summary

Error Rates In this activity, students explore calculations with simple rates and proportions, and basic time series data, in the context of news coverage of an important statistical study. From 1973 to 1995, a total of 4578 US death penalty cases went through the full course of appeals, with the result that 68% of the sentences were overturned! Reports of the study in various newspapers and magazines fueled public debate about capital punishment.

Share your modifications and improvements to this activity through the Community Contribution Tool »

Learning Goals

Students learn that when data presentations are drawn from popular media accounts, as opposed to standard statistical reports or textbook presentations, additional critical reading and thinking skills are required.

In the course of the activity, students will see how to

  • construct a simple tree diagram for computing proportions in two stages
  • pay attention to the numerator and denominator of a rate in context
  • estimate rate of change and average values on a linear time series plot
  • understand some pitfalls of using a double y-axis plot

Context for Use

This is an interactive lecture activity, with in-class discussion. In its most basic form, it is appropriate for an algebra-based introductory statistics class or a quantitative reasoning class following the introduction of basic descriptive statistics, including proportions and time series plots. Followup questions can be developed for more advanced classes.

At a brisk pace, covering just the basic calculations, the activity can be covered in about half a class period (10-15 minutes for set up and group discussion, 10 minutes for wrap-up). At a more leisurely pace, leaving time for side discussions, it could easily expand to a full period.

Description and Teaching Materials

I provide a handout summarizing the news sources, as described above, along with the following discussion questions. In small groups, the students respond to the questions. We then reassemble the whole class to clear up any misunderstandings and to pool insights.

Discussion Questions

  1. The Los Angeles Times published a lengthy article on the Liebman study ("Death Penalty Overturned in Most Cases. Justice: Study finds courts void execution more than two-thirds of the time."June 12, 2000, p. A1). We read there that "state courts initially overturned 47% of the death sentences, having found serious legal flaws. Later federal review discovered 'serious error'...in 40% of the remaining cases, resulting in an overall 68% reversal rate nationwide." How was this overall rate computed?
  2. The British journal The Economist published a series of articles on America's death penalty. "America's Death Penalty Lottery" (10 June 2000, p. 15), states: "Since 1976 (when the death penalty was reintroduced), America has executed 640 people (due to be 642 by the time you read this). So the 87 people exonerated after being sentenced to death amount to one reprieve for every seven killed. ...If an airline crashed once for every time it reached its destination, it would surely be suspended immediately." How do you reconcile the one-in-seven figure with the preceding Los Angeles Times summary?
  3. "Dead Man Walking Out" (The Economist, 10 June 2000, p.21) reports that "Since 1973, 87 prisoners have been released from death row after new evidence suggested they were innocent...And late acquittals are increasing. Between 1973 and 1993, an average of 2.5 death row inmates a year were found to be innocent. In the six years after that, the rate nearly doubled to 4.6." This passage is clearly meant to suggest that the error rate is increasing. But is it? Does the following graphic, reproduced from from the Los Angeles Times article, shed any light on the matter?
  4. In "Murder One" (17 June 2000, p.33) The Economist described the pervasiveness of the errors: "85% of the states which have the death penalty make serious mistakes in 60% or more of the cases. ...If there is any state that looks unusually bad it is Virginia: for its size it executes five times as many people as other states and reverses only a quarter the number of cases (see chart)." 
     
    What do you find confusing about this chart? What recommendations do you have for improving the presentation?

Teaching Notes and Tips

Notes on News Sources

To make the activity authentic, I like to provide newspaper clippings or printed excerpts. If you have access to Nexis/Lexis, you can find these online, but unforunately without charts and pictures. The graphics included here are my Excel reconstructions of the originals (I even tried to match the color scheme from the Economist). The articles referenced the Liebman report, and I found the raw data in the Appendices there:

Notes on Discussion Questions
  1. As Yogi Berra said (of baseball), "90% of this game is half mental." A simple tree diagram showing with two stages, one state and one federal, does the trick. Students can usually do this quickly.
  2. Computing a proportion is straightforward, but the one-in-seven figure is midleading. Sensible ratios might be the number of innocent defendants sentenced to death divided by the total number sentenced, or, more dramatically, the number of innocent people executed divided by the total number of executions. The one-in-seven figure is a mismatch of numerator and denominator: it divides the number of people released from death row by the number of executions. These points were cogently argued by MIT Professor Arnold Barnett in an op/ed piece.
  3. The time series from the LA TImes shows that the death row population grew approximately linearly over the study period. This makes it easy to estimate the average death row population from 1973 to 1993 (approx 1500) and from 1993 to 1999 (approx 3200). Since the population roughly doubled, with a constant error rate we would expect the number of errors to double. Students typically figure out that the total death row population matters. With some prodding, they understand how to estimate the value average of a linear function over different time periods.
  4. Double-y axis plots are notoriously misleading, because there are two separate scales. Nevertheless, Excel's Chart Wizard touts them as a "Classic comparison chart. Lines plotted on two axes." Many people's first reaction to the Economist graphic is that more than 100% of the sentences are being carried out! With some discussion, students realize that by tweaking the scales, one can make two variables look either highly correlated or nearly independent!

Assessment

I count activities like this as part of a classroom participation grade. Each group designates a recorder, who turns in a brief summary of the group's discussion at the end of class. I also expect active participation in discussion when the class reconvenes as a whole to pool insights.

For a homework exercise, I ask students to create their own versions of the graphs and summary statistics, in a way that avoids the points of confusion identified in class. In courses that use journaling, I ask for individual responses to the activity their journals.

In a writing-based first-year seminar course, one of my favorite assignments ask students to write critiques of three data graphics that they have culled from media reports (the only ground rule is that at most one can come from USA Today!).

References and Resources