Examples of QR Assessment Instruments
Are you interested in quickly assessing your numeracy skills? Try this 2-minute numeracy test to get quick feedback!
There are a variety of Quantitative Reasoning (QR) assessment instruments that have been developed by researchers interested in gathering information on and improving students' QR skills (as well as their attitudes towards QR). A sampling is included below, broken down into four categories:
- Generalized Critical Thinking Assessment Instruments that Include Quantitative Reasoning Skills.
- Generalized Quantitative Reasoning and/or Statistical Literacy Assessment Instruments.
- Subjective Numeracy Instruments.
- Health Literacy Assessment Instruments.
- Other Focused Assessment Instruments.
For assessment instruments developed at the course/instructor level, please see our page on resources developed as a result of this project.
(1) Generalized Critical Thinking and STEM Assessment Instruments that Include Quantitative Reasoning Skills
Collegiate Learning Assessment (CLA)
The CLA "presents realistic problems that require students to analyze complex materials and determine the relevance to the task and credibility. Students' written responses to the tasks are evaluated to assess their abilities to think critically, reason analytically, solve problems and communicate clearly and cogently."
Critical Thinking Assessment Test (CAT)
The CAT is a faculty-scored short-answer test and most of the questions are open-ended, including a number that focus on QR skills. In particular, the CAT questions assess how well students can: (1) summarize the pattern of results in a graph without making inappropriate inferences, (2) evaluate how strongly correlational data support hypotheses, (3) provide alternative explanations for a pattern of results that has many possible causes, (4) identify additional information needed to evaluate a hypothesis/interpretation, (5) evaluate whether spurious relationships strongly support a claim, (6) provide alternative explanations for spurious relationships, (7) identify additional information needed to evaluate a hypothesis/interpretation, (8) determine whether an invited inference in an advertisement is supported by information, (9) provide relevant alternative interpretations of information, (10) separate relevant from irrelevant information, (11) analyze and integrate information from multiple sources to solve a real-world problem, (12) use basic math skills to solve a real-world problem, (13) identify suitable solutions for a real-world problem using relevant information (14) identify and explain the best solution for a real-world problem using relevant information, and (15) explain how changes in a real-world problem might affect the solution.
The Student Assessment of Learning Gains (SALG) Instrument
"The Student Assessment of their Learning Gains (SALG) instrument was developed in 1997 by Elaine Seymour was she was co-evaluator for two National Science Foundation-funded chemistry consortia (ChemLinks and ModularCHEM) that developed and tested modular curricula and pedagogy for undergraduate chemistry courses. The SALG instrument focuses exclusively on the agree to which a course has enabled student learning. The SALG asks students to assess and report on their own learning, and on the degree to which specific aspects of the course have contributed to that learning. The instrument has been revised to include five overarching questions, each of which an instructor can customize through sub-items."
(2) Generalized Quantitative Reasoning and/or Statistical Literacy Assessment Instruments
Association of American Colleges and Universities (AAC&U) Quantitative Literacy Valid Assessment of Learning in Undergraduate Education (VALUE) rubric
See QL VALUE Rubric (Acrobat (PDF) 103kB Jan7 14). (The first page provides more details about the rubric.)
See also the Quantitative Literacy Assessment Rubric (QLAR (Acrobat (PDF) 111kB Jan7 14)), a slightly modified version of the AAC&U rubric.
"RiskLiteracy.org is a nonprofit university-based project designed to help increase awareness about risk literacy, i.e., the ability to accurately interpret and make good decisions based on information about risk. The site features the Berlin Numeracy Test and other interactive content, offering automated scoring and personalized feedback and training."
Carleton College's Quantitative Inquiry, Reasoning and Knowledge Initiative (QuIRK) Rubric for Assessing Quantitative Reasoning in Papers
This rubric is designed for use at the institutional level and is designed to assess four QR goals including whether students (1) think quantitatively, (2) implement QR competently, (c) interpret and evaluate QR thoughtfully, and (4) communicate effectively.
See: Carleton QuIRK Rubric (Microsoft Word 71kB May27 12)
Community College of Vermont QR Assessment
In addition to their coursework, students must obtain a satisfactory score on a Quantitative Reasoning (QR) assessment. Information about the assessment as well as practice problems are available online.
Evaluation and Assessment of Teaching and Learning about Statistics (e-ATLAS)
"The purpose of e-ATLAS is to develop a suite of high-quality assessment instruments that can be used to help evaluate the effectiveness of reform efforts associated with the teaching and learning of introductory statistics at the tertiary level. The two primary instruments develop as part of e-ATLAS are the Statistics Teaching Inventory (STI) and the Goals and Outcomes Associated with Learning Statistics (GOALS v.2). An additional instrument, the Basic Literacy in Statistics (BLIS), developed by Laura Ziegler as part of her doctoral dissertation research, has also been included in the instruments currently available for the e-ATLAS project. Each of these instruments can be administered online. Since summer 2013, these instruments have been used to gather large quantities of data. Although the funding period for e-ATLAS is ending, we are pursuing ways to continue to administer these instruments and to make them accessible to the statistics education community."
Graduate Record Examination Quantitative Reasoning Section
The GRE "is a standardized exam that is administered as part of the graduate school admissions process." The exam includes three sections: (1) verbal reasoning, (2) quantitative reasoning, and (3) analytical writing.
Hollins University QR Assessment Instrument
Hollins has a Quantitative Reasoning assessment instrument on which students must obtain a satisfactory score (or they must enroll in a course entitled, "Introduction to Quantitative Reasoning").
See the study guide here.
Insight assessment has a variety of numeracy assessments including (1) a The Business Critical Thinking Skills Test Numeracy, BCTST-N, (2) The California Critical Thinking Skills Test Numeracy, CCTST-N, (3) The Health Sciences Reasoning Test Numearcy, HSRT-N, (4) The Test of Everyday Reasoning Numeracy, TER-N, and (5) The Quant Q Test. Each of these tests is described in more detail at the pages linked.
James Madison University Quantitative Reasoning Assessment Instrument
This instrument is a "computerized, multiple-choice test developed by the JMU Center for Assessment and Research Studies (CARS) and faculty from science and mathematical domains. It is designed to assess the quantitative reasoning of students who have completed their general education requirements, and measures the following learning objectives: (1) use graphical, symbolic, and numerical methods to analyze, organize, and interpret natural phenomena; and (2) discriminate between association and causation, and identify the types of evidence used to establish causation." This test is recommended for use with general education program assessment and evaluation.
Lehman College
A generalized QR instrument that focuses on both QR attitudes and skills, developed by Esther Wilder. See: EI Wilder Lehman College QR Assessment (Acrobat (PDF) 284kB Jan7 14)
Levels of Conceptual Understanding in Statistics (LOCUS)
"Locus is an NSF-funded DRK12 (DRL-1118168) project focused on developing assessments of statistical understanding. These assessments will measure students' understanding across levels of development as identified in the Guidelines for Assessment and Instruction in Statistics Education (GAISE). The intent of these assessments is to provide teachers, educational leaders, assessment specialists, and researchers with a valid and reliable assessment of conceptual understanding in statistics consistent with the Common Core State Standards (CCSS)."
Milo Schield's Statistical Literacy Tests
Milo Schield (Augsburg College) has a statistical literacy assessment instrument that focuses on reading graphs as well as a test on describing and comparing percentages.
This numeracy quiz is part of a project designed to "improve quantitative awareness, critical analysis, and evidence-based perspectives among journalism students and journalism instructors." This initiative is led by Michael Ranney at the University of California/Berkeley.
Norfolk State University (NSU)
Quantitative Reasoning is defined to encompass four skills for purposes of assessment including: "(1) Numeric or arithmetic contexts: Estimation and approximation, percent, ratio and proportion, simple and compound interest, simple formulas, etc.; ( 2) Conceptual contexts: Pattern recognition, symbolizing data, graphing analysis, algebraic expressions, equations, models, etc.; (3) Geometric contexts: Measurement, conversion of units, shapes and sizes, basic relationships among lines, angles, triangles, polygons, etc., and (4) Data representation and chance element contexts: Counting techniques, data distribution, basic statistical measures, elementary probability, etc." NSU has "adopted a course-embedded direct approach to assess the competency of NSU students on the four quantitative reasoning dimensions." To gather evidence of quantitative reasoning competency, the faculty developed the NSU Quantitative Reasoning Test (QRT) and they also rely on some indirect measures of assessment (e.g., the National Survey of Student Engagement).
At Northern Essex, "the institutional co-chairs created a uniform assignment that would tap into the abilities described in two outcomes developed by the Core Academic Skills Assessment Committee, named Global Awareness and Quantitative Reasoning. The assignment created was a scenario which presented a real-world problem faced by the United States; a graphical display of information relevant to solving the problem; and brief descriptions of situations in six fictional countries including information on political, economic and cultural factors." The results are available online.
Quantitative Literacy and Reasoning Assessment (QLRA) Instrument
This is a non-proprietary QLRA instrument and is a compilation of the Bowdoin, Wellesley and Colby Sawyer exams. As described by Eric Gaze at Bowdoin College, PI on an NSF-funded QLRA grant, this instrument "is being piloted at schools across the country, including community colleges, public universities and liberal arts schools." Over 1600 students have taken the test, with mean scores at schools ranging from 39% to 79%.
University of Akron Quantitative Reasoning Assessment
As noted on U-Akron's web site, "During the 2010 – 2011 academic year, faculty coordinating College Algebra and Statistics for Everyday Life (SEL) performed an assessment of students' ability in quantitative reasoning and mathematical reasoning."
University of Arizona (Kate Follette and Donald McCarthy's) QR Assessment Instrument
This is a multiple choice QR assessment instrument that was developed for the purposes of assessing the extent to which quantitative skills can be improved through a semester of introductory undergraduate science instruction.
University of Arkansas (Shannon Dingman and Bernard Madison's) QR Assessment Instrument
This is a multiple choice QR assessment instrument used in a foundational QR course at the University of Arkansas.
See the appendices here
University of Virginia QR Assessment
A faculty committee representing major disciplines and each undergraduate school developed an "in-house" instrument to assess quantitative reasoning. The reports are available at the above web site.
Wellesley College QR Assessment Instrument
Wellesley College has a Quantitative Reasoning assessment instrument for which students must obtain a satisfactory score on (or enroll in a course entitled, "Introduction to Quantitative Reasoning"). As described in their student study guide, "The QR Assessment tests your quantitative skills, including your ability to read and understand information presented in formulas, tables, and graphs; to interpret information and draw appropriate inferences; and to solve real world problems that deal with numbers or data. The mathematical skills you will apply on this test span arithmetic, algebra, graph reading, geometry, and linear and exponential modeling. Logical and statistical skills are core 'QR skills.'"
See the study guide here.
WizIQ Quantitative Reasoning Tests
WizIQ includes a number of resources for online education, including a unique platform. Some of the tests for quantitative reasoning are particularly useful.
(3) Subjective Numeracy Instruments
This initiative resulted in a survey that assesses students' attitudes towards QR and mathematics. See pages 29-30.
Human Resources and Skills Development Canada
Human Resources and Skills Development Canada has a numeracy self-assessment which focuses on individuals' attitudes and self-described ability to undertake a variety of quantitative tasks.
The math anxiety scale is a 13-item self-reported scale to assess math anxiety (Rolison et al. 2016).
The Science Education for New Civic Engagements and Responsibilities (SENCER) Student Assessment of their Learning Gains (SALG) focuses on STEM overall, but had much applicability to QR. As described on their website, "The assessment tool also asks students to report on their science skills and interests, as well as the civic activities in which they engage."
"An 8-item measure, the Subjectve Numeracy Scale (SNS), was developed through several rounds of testing. Four items measure people's beliefs about their skill in performing various mathematical operations, and 4 measure people's preferences regarding the presentation of numerical information" (Fagerlin et al. 2007).
(4) Health Numeracy Assessment Instruments
A four-item asthma numeracy questionnaire (Apter et al. 2006).
The Diabetes Numeracy Test is "an assessment test designed to investigate the numeracy skills in patients with diabetes." For more information about the test, see Huizinga and associates (2008).
General Health Numeracy Test (GHNT)
A general health numeracy test (Osborn et al. 2013).
Medical Data Interpretation Test
The medical data interpretation test is "a reliable and valid measure of the ability to interpret medical statistics" (Schwartz et al. 2005).
Numeracy Scales
A variety of numeracy instruments including a 3-item (chance, proportions and percentages) (Schwartz et al. 1997), an 11-item (risk and fractions as well as chance, proportions and percentages) (Lipkus et al. 2011), and a 15-item test (base rates and complex likelihood calculations as well as risk, fractions, chance, proportions, and percentages) (Peters et al. 2007).
The Newest Vital Sign assesses general literacy and numeracy skills as applied to health information, yielding an overall estimate of health literacy. Developed by Pfizer.
STAT-Interest and STAT-Confidence
An instrument that measures perceived ability to understand and interpret medical statistics (Woloshin et al. 2005).
The Test of Functional Health Literacy in Adults (TOFHLA) -- Numeracy Component
The TOFHLA consists of a 50-item reading comprehension and a 17-item numerical ability test, taking up to 22 minutes to administer.
The Test of Functional Health Literacy in Dentistry (TOFHLiD)
The TOFHLiD is an instrument designed to measure functional oral health literacy (Gong et al. 2007).
(5) Other Focused QR Assessment Instruments
Lehman College Sociology Program
As part of an initiative to infuse data analysis into the sociology curriculum at Lehman College (NSF DUE 0411401], 3 versions of a QR assessment instrument were developed (PI Esther Wilder) to assess a wide range of basic mathematical and QR skills including measures of central tendency, percentages, ratios, and reading tables and graphs.
See: Lehman Sociology Version A (Microsoft Word 207kB May27 12), Lehman Sociology Version B (Microsoft Word 81kB May27 12) and Lehman Sociology Version C (Microsoft Word 86kB May27 12).
Kathy Schrock's Infographics as Creative Assessment
This page includes a variety of resources for assessing infographics, including an assessment instrumented created by Kathy Schrock as well as numerous additional ones.
Sources Cited
Apter, Andrea J., Jing Cheng, Dylan Small, Ian M. Bennett, Claire Albert, Daniel G. Fein, Maureen George, and Simone Van Home. 2006. "Asthma Numeracy Skill and Health Illiteracy." J Asthma 43(9): 705-710.
Fagerlin, A., Zikmund-Fisher, B.J., Ubel, P.A., Jankovic, A., Derry, H.A., & Smith, D.M. 2007. "Measuring Numeracy without a Math Test: Development of the Subjective Numeracy Scale (SNS)." Medical Decision Making 27(5): 672-680.
Gong, D.A. J.Y. Lee, R.G. Rozier, B.T. Pahel, J.A. Richman, and W.F. Vann, Jr. 2007. "Development and Testing of the Test of Functional Health Literacy in Dentistry (TOFHLiD)." Journal of Public Health Dentistry 67(2): 105-112.
Huizinga, Mary M., Tom A. Elasy, Kenneth A. Wallston, Kerri Cavanaugh, Dianne Davis, Rebecca P. Gregory, Lynn S. Fuchs, Robert Malone, Andrea Cherrington, Darren A. DeWalt, John Buse, Michael Pignone, and Russell L. Rothman. 2008. "Development and Validation of the Diabetes Numeracy Test (DNT)." BMC Health Services Research 8: 96.
Lipkus, Isaac, Greg Samsa, and Barbara K. Rimer. 2001. "General Performance on a Numeracy Scale among Highly Education Samples." Medical Decision Making 21(1): 37-44.
Madison, Bernard L., Stuart Boersma, Caren L. Diefenderfer, and Shannon W. Dingman. N.d. "Quantitative Literacy Assessment Rubric." Unpublished document.
Osborn, C.Y., K.A. Wallston, A. Shpigel, K. Cavanaugh, S. Kripalani, and R.L. Rothman. 2013. "Development and Validation of the General Health Numeracy Test (GHNT)." Patient Education and Counseling. [Epub ahead of print]
Peters, Ellen, Nathan Dieckmann, Anna Dixon, Judith H. Hibbard, and C.K. Mertz. 2007. "Less is More in Presenting Quality Information to Consumers." Medical Care Research and Review 64(2): 169-190.
Rolison, Jonathan J., Kinga Morsanyi, and Patrick A. O'Connor. 2016. Can I Count on Getting Better? Association between Math Anxiety and Poorer Understanding of Medical Risk Reductions." Medical Decision Making 36(7).
Schwartz, Lisa M., Steve Woloshin, William C. Black, and H. Gilbert Welch.1997. [link http://www.ncbi.nlm.nih.gov/pubmed/9412301 '"The Role of Numeracy in Understanding the Benefit of Screening Mammography." Annals of Internal Medicine 127(11): 966-972.
Schwartz, Lisa M., Steve Woloshin, and H. Gilbert Welch. 2005. "Can Patients Interpret Health Information? An Assessment of the Medical Data Interpretation Test." Medical Decision Making 25(3): 290-300.
Woloshin, Steve, Lisa M. Schwartz, and H. Gilbert Welch. 2005. "Patients and Medical Statistics. Interest, Confidence, and Ability." Journal of General Internal Medicine 20(11): 996-1000.