Rubric for Evaluating Quantitative Reasoning in Student Writing

Developed by Carleton College's Quantitative Inquiry, Reasoning, and Knowledge (QuIRK) initiative

Purpose

We apply the rubric to our sophomore writing portfolio to evaluate the degree to which our quantitative reasoning (QR) initiative is altering the nature of student arguments. Because the writing portfolio does not require students to submit papers involving QR, we cannot possible use this instrument to evaluate individual students; it is designed to assess at the programmatic level. (While limiting in some contexts, it does allow us to study whether our initiative's work is changing the propensity of our students to call upon QR skills in unprompted circumstances.)

Implementation

Each year, a team of roughly 12 faculty and staff members read a sample of student papers. In total, we aim to read about 400 papers over 3 days.

Use of the Data

The resulting data are used to assess the initiative (to satisfy both granting agencies and accreditation committees), to guide the design of professional development activities, and to motivate change on campus.

Effectiveness

The rubric has surely succeeded as a formative tool. Faculty who engage in the assessment of student work see first-hand how we are succeeding and failing with our students. This is an incredible motivation for pedagogical and curricular improvement.

What is less clear is whether the data can be used for summative assessment. This year we revised the rubric to improve inter-rater reliability. In the next two years we will be using the improved rubric to generate data which we will combine with transcript data to see if we can say useful things about the interaction between curricular changes and student performance.

Instrument

Download the rubric. (Microsoft Word 80kB Feb3 09)