Charles Stewart III, Political Science

Initial Publication Date: October 25, 2013

Charles Stewart III is a professor of Political Science at MIT, a private 4-year research university. Information for this case study was obtained from an interview conducted on August 6, 2013. This page is part of a collection of profiles about a variety of techniques for integrating Quantitative Reasoning (QR) across the curriculum.

Jump down to Design and Implementation of QR Goals | Key QR Assignment of the Course | Challenges | Advice | Documents

Overview and Context

About the Course

The Political Science Laboratory course is required of all political science students. We strongly urge them to take it at the end of the second semester of their junior year as part of a sequence that eventually leads them to writing a thesis in political science. The number of students in the class is usually 12, but can be up to 16. I've taught this course in various forms for almost 25 years.

The core of the class is a set of statistical techniques, and so in that sense it looks like a statistics class. The Hard-Nosed Empiricist is an assignment done on a daily basis that helps students learn to analyze claims. Students are also required to write a term paper where they think about a question in the political world and interrogate it quantitatively.

Syllabus for Political Science Laboratory (Microsoft Word 87kB Sep6 13)

Key QR Assignment Description (links to section in this page)

How Quantitative Reasoning (QR) and Literacy are Approached

In my mind, the phrase quantitative reasoning suggests the synthesis of quantitative concepts across a wide range of problems. We explicitly emphasize the importance of developing this ability throughout both undergraduate and graduate training. However, we generally use the term, "quantitative literacy" rather than "quantitative reasoning" as an umbrella term to cover both quantitative skills and the higher order thinking associated with solving problems.

Design and Implementation of QR Goals

Motivation to integrate QR

Our department's motivations were to have our students be known for their ability to solve problems and to be able to analyze empirical claims. In addition, QR was incorporated into this class in order to meet a general MIT requirement.

At MIT our comparative advantage in the world is turning out students who know how to solve problems. We want our students to be able to apply quantitative reasoning and skills to help them to address any problem using the tools of social science.

We also want our students to be able to confront empirical claims that they believe themselves, read in the newspapers or watch on television. We want to give them the capacity to think critically about these empirical claims and be able to verify for themselves whether or not the claims are valid.

This course was designed in the mid-1960's as a way of helping political science students meet the MIT laboratory requirement. Traditionally, all students took a science and engineering lab, but as the curriculum evolved, the various departments started their own labs that were more pertinent to their students, which is why this class is called, "Political Science Laboratory."

QR goals

The quantitative reasoning goals include having students consider an empirical claim that is made about a social phenomenon and recognize what the causal implications are of that claim. We focus on research design as a way of parsing claims. Then we move to the next step, which is to imagine the different ways in which data could be gathered to test the empirical implications, and perform the test. Finally, students are expected to synthesize these ideas and communicate their quantitative findings through writing a final term paper.

Pedagogic approaches used

If you were to observe the class you would see two sets of things, one is you would see the class sitting around the table listening to me lecture about some topic with a fair amount of interruptions, which are welcome. The lectures are mostly about issues in research design and statistics.

The second thing you would see is the class doing hands-on assignments such as importing data sets into statistical software, making graphs, and running statistical techniques. I have discovered that doing these types of assignments with the professor circulating around the room and helping problem-solve is a better way to teach students how to become independent compared to letting them figure it out on their own.

I have found that in teaching students how to do empirical work, oftentimes it is the logistics that get in the way of students being successful. Students are often overconfident in their ability to do computer-related assignments and tend to wait until the last minute and not do it very well. Since I think that doing the empirical work is very important, I take the time to work through these hands-on assignments before asking them to use the tools to do research.

Knowing the course is successful

I measure the success by the quality of the papers they write at the end. With very few exceptions, the students taking the class have never done a project like this before. So where they end up at the end of the semester is a consequence of both the inputs and their own application during the semester.

By the end of the semester, I feel confident almost all of them would be able to write a thesis or, at the very least either verify or refute claims based on data.

Additionally, in the process of helping students informally with their papers, I notice the types of help that they need and this gives me a sense of both what I could improve in my teaching and the success of the course.

Key QR Assignment of the Course

The main idea of the Hard-nosed Empiricist assignment is to teach students to think and act like a social science researcher. More broadly, it is attempting to enable them to confront empirical questions that they believe or read about and to figure out ways to independently verify these claims. They must find the data, think about what a reasonable test of the claim would be, and perform the test.

Hard-nosed Empiricist Student Handout (Links to section on this page)

The semester is divided in such a way that for the first two-thirds it is more or less a traditional class with some lecture and a variety of techniques (e.g. hands-on exercises.) The Hard-nosed Empiricist assignment is done in the first 15-20 minutes of class during this period of the semester. This assignment is described in more detail below.

The first part of the Hard-nosed Empiricist assignment is a homework assignment. Students must find empirical claims that they can study--for instance, "bad weather causes voter turnout to go down." Then they search for research someone else has done to address the claim, which they will present to the class.

Students must clear with me ahead of time both the empirical claim and their example of research to verify the claim. I want the research they select to be either trustworthy research or a good example of bad research.

Students examine the research, present it to the class, and tell us what they think it says. They lead the class in a mini discussion about the design of the study that is verifying the claim. They point out whether or not there were certain features such as internal or external validity that would lead us to trust or not trust the results of the research.

Although the students lead the discussion, I guide them with questions and make sure they're clear about why they chose the example, why they think the claim and the research are related to one another, and what they think are the threats to validity.

Part of my guiding the discussion is to try to get them to imagine, "In the best of all possible worlds, what would you want to do? Given what is available, what can you do?" I try to guide them in a way that inspires them to do better research or to think harder about the questions. More importantly perhaps, I want to help them understand what the limits to the research are short of just dismissing any empirical work in the social sciences.

Doing this on a daily basis helps the students learn how to dissect claims and make arguments. It also provides regular practice with quantitative reasoning and helps them write their term paper at the end of the semester.

The last third of the semester is a series of workshops where students work on term paper projects. They must form the question, gather the data and make an argument as one would in a professional journal. For this last third of the semester, the class meetings are either informal--small group or individual problem-solving--or the students make presentations about their projects and receive feedback from other class members. At this point in the class, the students have internalized the concepts from the earlier smaller exercises such as the Hard-nosed Empiricist, so that are able to provide constructive feedback to help each other.

Challenges

  • Helping students to accept challenges to their beliefs. Students often have an unwillingness or a difficulty accepting empirical findings when they disconfirm a previously held world view. Students generally take political science classes because they believe things strongly about politics or about the social world. But one of the things that often happens when they get serious about subjecting those beliefs to quantitative tests is they find that their beliefs were probably wrong. I think the biggest struggle is in getting students to internalize what they find quantitatively in their general beliefs about the world. This leads to problems writing up the report because their writing gets convoluted, they continue to claim that their belief is correct when the sign is in the opposite direction or ambiguous, or they come up with really complicated post hoc rationalizations about why they got the results they did.
  • Setting the correct tone for the assignment. The first few students who do the Hard-nosed Empiricist assignment set the tone for the other students. Students take clues from one another about what's acceptable, which I've discovered over the years. Sometimes even when I've approved the claim and the research to back it up, the presentation is done in a half-hearted way. If that happens in the beginning of the semester, then it influences the others. But if I have a good exemplar in the beginning of the semester, then other students are motivated to also make a strong effort.

Advice

  • Just do it. Number one piece of advice.
  • If you do nothing else, have students independently gather data. Depending on the topic of your class, do an exercise that has students gather original data and do something with it--whether it be a table or a scatter plot--something really simple. Because students love doing it and it's a way of opening the world of quantitative analysis to them.
  • Start simple. Rather than having students do library research where they basically re-repeat what others have done, have them go to a book or the website with a table of data. For instance, maybe the data looks at the averages between the red states and the blue states--between places with Democrats and Republicans--a very simple comparison. It doesn't have to be very complicated, but is a great place to start.
  • Don't worry about teaching statistical theory. Don't worry about things like the theory of statistical estimation. I think that one of the traps that I've discovered in political science is a belief that in order to do quantitative reasoning you have to start with things like sampling theory. Some say you don't want students to develop bad habits about statistical inference and the rest, so you have to start very abstract. But I think that's exactly wrong. Students get into political science in our classes because they're motivated by the empirics. You have to deal with motivation first. So that's why I recommend doing really simple things because it opens the door to the students and is really easy for faculty to do.
  • Use Repetition. Having taught a class like this for a quarter of a century, I'm very aware that you can't tell students how to do something in the quantitative reasoning realm, and then expect them to just do it. You have to keep coming back to it and reminding them and catching them when they make errors.

Documents