Tim Kubal, Sociology
Jump down to Design and Implementation of QR Goals | Key QR Assignment of the Course | Challenges | Advice
Overview and Context
About the Course
This is an upper-division sociology course titled, Sex and Gender, which I have taught for over ten years. During this time, I have shifted in my approach from being a primarily qualitative sociologist to incorporating a substantial amount of quantitative methods and reasoning into my courses and research.
The student audience is primarily composed of non-majors because this course satisfies a general education diversity course requirement. The course is an elective for sociology majors. Class size is generally 50 to 60 students.
Key QR Assignment Description (links to section in this page)
How Quantitative Reasoning (QR) and Literacy are Approached
I have found that incorporating quantitative literacy and reasoning into my courses leads to greater student engagement. I have shifted from primarily being a qualitative researcher to being a mixed methods sociologist, and this approach has been reflected in my teaching. In my opinion, it is important for students to understand substantive concepts within a quantitative context.
Design and Implementation of QR Goals
Motivation to integrate QR
My motivation for integrating quantitative reasoning grew out of a desire to increase student engagement and learning. When I first started teaching at the college level, I was "the sage on the stage." I was primarily conveying information through lectures. Over time I became bothered with that approach because of how the students were reacting. They seemed disengaged and were not learning a lot.
My motivation for integrating quantitative reasoning increased when I became involved in the American Sociological Association's program called Integrating Data Analysis (the IDA program.) I decided that I wanted to incorporate some quantitative exercises into my courses in order to engage the students and excite them about the material, encourage them to read more, and ultimately, to learn more.
QR goals
My goal is to create excitement around analyzing the world with numbers. I want to encourage engagement, particularly since it is primarily a general education course.
I also want them to practice and get comfortable with numbers and tables and statistics. For this particular class we use Chi-squared analysis almost exclusively, but I do not require them to know how or why it is computed, only how to interpret the statistic. I do not teach them advanced methods or statistics lingo, because that is not the purpose of this course.
In terms of sociology content, I want students to learn how men and women are different from one another, statistically speaking. We primarily use public opinion polls, and I want students to understand the power and the limitations of data gathered in this way.
Pedagogic approaches used
Our students are more likely to succeed when they learn by doing rather than by only hearing. For that reason, I emphasize active learning in order to engage students and also to encourage them to be responsible for their own learning.
In order to do the interview project described below, I scaffold students' learning by first providing them with training in doing interviews, building their listening skills, and using the data collection tools and technology. I bring in a student assistants to help during this training, which allows for more one-on-one assistance. I also employ a student to organize interview recordings so that students can hear what they sound like and can use this information to improve their interviewing skills.
Since the class is too large to occupy a computer lab, I use screen captures to teach students how to use the course software and to analyze data. The screen capture with audio shows my mouse and where I click. Students can watch the screen capture over and over, and this appears to work well.
Knowing the course is successful
To evaluate the course, I use an educational survey that measures learning outcomes in relation to the methods and topics taught in the course. The survey asks questions such as, "How much did you learn about statistics? How much did you learn about qualitative methods? How often did you take part in the assignment? How much work did you do?" I have been using this evaluation for five semesters and have collected hundreds of student responses.
I have analyzed this survey data and found that the phone survey assignment exercise was the only variable that had any real influence on important educational factors including learning outcomes, student engagement, and student satisfaction.
Key QR Assignment of the Course
General Social Survey (GSS) data assignment. I set up a short online data analysis task every week of the 16-week semester, and they must do 8 of them. The purpose is to explore variables related to the weekly readings on topics such as education and gender. Students use an online tool called Survey Documentation and Analysis (SDA), which is a data analysis tool created at Berkeley.
More about the GSS data assignment
HidePhone Survey assignment. The larger quantitative reasoning assignment involves students collecting data from phone interviews, analyzing the data, and writing a summary paper.
For this assignment, students use a Computer-Assisted Telephone Interviewing (CATI) lab that I developed to interview members of the public. Students can call in from anywhere using their cell phones. When they call in, the system automatically helps them connect with an interviewee using a randomly generated phone number. The system also shows them the interview protocol and automatically records the conversations.
We spend the first three or four weeks training students how to do interviews, how to deal with public opinion, and how to handle ethical issues. They practice by interviewing each other quite a few times. Then, usually by about the fourth week, they start doing the interviews and continue for a month.
The interview protocol uses mixed methods, primarily quantitative closed-ended questions that are mostly taken from the General Social Survey. We chose questions from existing surveys so that we could have good comparisons. We also include a couple of open-ended questions where we ask about social topics--crime, homelessness, healthcare, and food insecurity. We ask interviewees about their views on the extent of the problem, the cause, and the solution. In the Sex and Gender class, we look to see how men and women answer these questions differently.
Students transcribe the qualitative part of the interviews, which amounts to about 4 to 5 pages of writing per student. To make transcribing easier, I teach them how to use software including Dragon Natural Speaking and Windows speech recognition. I use screen captures to train students to use this software, as for the other software packages we use.
To analyze the qualitative data students use NVivo, a text analysis software program. I provide students with a suggested coding schema, but they may also develop their own coding schema if they prefer.
To analyze the quantitative data students use the statistical program SPPS. They must compare gender and some other variable of their choosing. They look for a statistically significant difference between responses given by men and women. Again, I teach them the software by doing a screen capture with audio. In this way, they can follow along and produce a table with a Chi-squared statistic without having to learn much more about how to use SPSS.
At the end of the semester, students turn in a ten-page paper that has several sections, including both quantitative and qualitative analysis. For the qualitative part, they look for patterns in how people answer the questions. For the quantitative part, they include three or four Chi-squared tables plus short written interpretations.
Challenges
- Student push-back. Some people either may not like numbers or they are not well enough prepared quantitatively speaking, and so it is really hard to reach them. They do not ask questions or seek help and so they do not seem to improve. However, most students really enjoy the phone survey project because they use data that they have collected, which is meanful for them.
- Grading. Grading is time-consuming and can be tedious. I prefer grading the qualitative analysis because it is usually more creative. Grading the quantitative analysis can be dull due to the repetitive nature of the responses.
Advice
- You don't need to give up qualitative approaches. I was trained and completely adopted the identity of myself as a qualitative researcher when I graduated with my PhD. Over the last ten years, I've transformed. I've retooled. I've become much more quantitative--I'm a mixed methods person now. It's made me a better teacher. It's made me a better scholar, and I know that it's allowed me to teach my students in a much more comprehensive way.
- Start slow and expand. Start with doing one simple thing well and then gradually add components over time.
- Use online resources. There are lots of useful resources on the web:
- Social Science, Research, and Instructional Council (SSRIC) has online textbooks, modules and data sets.
- Inter-University Consortium for Political and Social Research (ICPSR) has recently introduced an online data analysis program. They have adopted the SDA online web-based analysis, so almost all of their datasets are available in this way. You can go to the ICPSR website and get access to hundreds of datasets and analyze them right on the web.
- The Roper Center is an online database of public opinion polls and data. When you want to create survey questions, it's always a good place to look. You can find questions that have already been used, and of course, data to analyze from previous surveys.