Assessing Quantitative Reasoning
Evaluating the Effectiveness of One's Own Teaching
![[reuse info]](/images/information_16.png)
Assessing student learning is a vital component of any initiative to improve students' QR/QL skills. The purpose of assessment is to gather information on the effectiveness of pedagogy (e.g., readings, assignments, exercises, etc.) to inform and/or improve one's teaching. Assessment instruments can take many different forms, ranging from multiple choice exams to rubrics (i.e., scoring guides/maps) to combinations of different approaches. Many preexisting QR assessment instruments that have been developed are structured as multiple choice tests and are designed to be used at the institutional or program level.
At the same time, multiple choice questions are not universally appropriate. Shannon Dingman and Bernard Madison (2010: 8) note that "the major challenge in assessing QL concerns the central goal of transfer of knowledge and cognitive processes to contexts that are unpredictable and of unbounded variation." Some QR-related skills (e.g., the ability to write about and communicate quantitative information) do not easily lend themselves to multiple choice assessment instruments. That being said, rubrics or scoring guides/maps are another increasingly popular approach for assessing questioning reasoning skills. A more detailed discussion of how to develop a rubric is presented here.
Our page on QR assessment instruments provides examples of a variety of QR assessment instruments. Our page of resources developed as a result of this project shows examples of QR assessment instruments designed for use at the course level.
What is Assessment?
As Linda Suskie (2009) notes, there are several core components of the assessment of student learning including:
- Establishing clear, measurable expected outcomes of student learning,
- Ensuring that students have sufficient opportunities to achieve those outcomes,
- Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations, &
- Using the resulting information to understand and improve student learning.
Quantitative Reasoning Assessment
An excellent resource for assessing QR is a 2010 edited volume by Bidgood, Hunt and Jolliffe entitled, Assessment Methods in Statistical Education: An International Perspective. Many of the articles in Numeracy also address QR assessment (see, e.g., Sundre and Thelk 2010).
As Davies and Marriot (2010: 12) note, in devising/adapting assessment instruments, careful consideration should be given to:
(1) the learning outcomes and the capabilities/skills (implicit or explicit) they imply,
(2) methods of assessment that match these outcomes and skills,
(3) the relative efficiency of different methods in terms of student time and staff time,
(4) the advantages and disadvantages of the various assessment methods, and
(5) appropriate marking schemes and criteria.
Context Matters
Assessing QR skills is necessarily shaped by context. Thus, for example, the level at which the assessment is being undertaken (e.g., institution, program, course, etc.) and the kind of course and/or programs (e.g., humanities vs. social sciences) where the assessment is being undertaken will strongly influence the kinds of assessment instruments that are developed and how they are used. Thus, for example, a QR assessment instrument that is focused at the institutional level and addresses a broad QR skills set will be very different from a QR assessment instrument that a faculty member develops who is integrating QR into a course on Shakespeare.
Milo Schield (2010) notes that approaches to assessing statistical literacy have often focused on four areas including:
(1) evaluating statistics in news stories,
(2) estimating quantities or making decisions in open-ended situations,
(3) describing and comparing statistics presented in graphs or tables, and
(4) answering multiple-choice questions on specific aspects of statistical literacy.
Other assessment efforts have evaluated students' attitudes and comfort with regard to quantitative reasoning (see Korey 2000).
How to Develop a QR Assessment Instrument for Measuring Student Learning
The following approach is important for developing a QR assessment instrument for measuring student learning:
(1) Articulate your QR learning goal(s).
(2) Design effective teaching tools to meet those QR learning goals (classroom activities, assignments, demonstrations, etc.).
(3) Adapt and/or develop an assessment instrument (or instruments) to ascertain whether or not you have met the QR learning goal(s) you have articulated. (The assessment instrument should be independent of the teaching tools, including assignments, you employ to meet your QR learning goals.)
(4) Develop an assessment plan that will effectively utilize the assessment instrument(s) (e.g., pretest/posttest, etc.) you will be using.
(5) Analyze the assessment data and consider how to "close the loop" and use the assessment results to improve instruction.
Sources Cited
Association of American Colleges and Universities. 2010. Quantitative Literacy VALUE Rubric. Washington, DC: Association of American Colleges and Universities.
Bidgood, Penelope, Neville Hunt and Flavia Jolliffe (Editors). 2010. Assessment Methods in Statistical Education: An International Perspective. Hoboken, NJ: John Wiley & Sons Ltd.
Davies, Neville and John Marriott. 2010. "Assessment and Feedback in Statistics." In Assessment Methods in Statistical Education: An International Perspective, edited by Penelope Bidgood, Neville Hunt and Flavia Jolliffe. Hoboken, NJ: John Wiley & Sons. Pp. 3-19.
Dingman, Shannon W. and Bernard L. Madison. 2010. "Quantitative Reasoning in the Contemporary World, 1: The Course and Its Challenges." Numeracy 3(2): Article 4.
Korey, Jane. 2000. "Dartmouth College Mathematics Across the Curriculum Evaluation Summary: Mathematics and Humanities Courses." Unpublished manuscript.
Schield, Milo. 2010. "Assessing Statistical Literacy: Take CARE." In Assessment Methods in Statistical Education: An International Perspectives, edited by Penelope Bidgood, Neville Hunt and Flavia Jolliffe. Hoboken, NJ: John Wiley & Sons. Pp. 133-152.
Sundre, Donna L. and Amy D. Thelk. 2008. "Advancing Assessment of Quantitative and Scientific Reasoning." Numeracy 3(2): Article 2.
Suskie, Linda. 2009. Assessing Student Learning: A Common Sense Guide (2nd ed.). San Francisco: Jossey-Bass.