Assessing and Programming for Quantitative Reasoning: Common Barriers, Common Assets

• Our campus does not have a good track record of communicating about potential academic initiatives or exploring them. There is very little campus-wide communication that is effective on our campus, and a QR program that would involve all divisions and departments would require complex communications that are not common. In most cases, change is not easily accepted here, particularly if it requires additional work or effort of faculty. This means that any plan or approach to start talking about a new initiative such as QR receives immediate negative responses. Our faculty view academic freedom and complete autonomy of teaching as their highest priority and a common means to assess QR could be viewed as a degradation of such a priority. So, for us this becomes a complex political issue.
• While improving the QR skills of our students is not in principle controversial, the impact that a new academic initiative could have on our programs is bound to be controversial for at least some members of our campus community. In particular, it is not clear that everyone will consider an ethos of QR as important as an ethos of writing.
• The barriers are that there are only pockets of interest and awareness of the value of QR across the curriculum and that we have limited staff and development resources to move forward on developing courses, programs and assessment practices.

Neil Lutsky (Carleton College) responds: I believe the central response to the need to inspire faculty buy-in is to mount an extended persuasive effort directed at a local academic community. Faculty are extremely busy, but they also care about the education they are providing students. Thus, faculty need to be shown that incorporating quantitative reasoning in their teaching, even in small measures, will contribute to their students' success in life. Furthermore, they need to recognize how developing their own facility with quantitative reasoning will yield personal benefits, for example, when faculty navigate quantitative information in prominent newspapers or in their professions. In sum, faculty need to be convinced that teaching quantitative reasoning is worth doing and falls within their abilities to do. Finally, faculty need to see that colleagues just like them have embraced the QR agenda, have enjoyed learning more about QR, and are experimenting with means of teaching QR. You don't need to inspire an entire community at the outset, but a committed group of faculty can have a powerful impact on a community by sharing their enthusiasms and experience.

• The fundamental challenges are time and people. Our campus has experience waves of retirements and is shifting from a teaching-focused institution to one that emphasizes more research.
• There is little resistance to implementing and extending practices to develop and assess QR programming, but we do not know how to do this. There is a need to create a time and space for reflective and informed discussions about these issues and that is not always easy when people are as busy as they already are. We do not see how a new QR center/initiative would be supported.
• Our faculty are already overbooked.
• The difficulty will be in recruiting faculty to do something new when they are already busy pursuing their own efforts.

Nathan Grawe (Carleton College) responds: There's no use denying it. Faculty time is in limited supply. To mitigate this concern, you might try to cast your QR initiative as helping faculty do better what they are already doing rather than as asking them to do something new. For example, if you have an active writing program, see if you can get faculty to think about teaching students to write better arguments by incorporating quantitative evidence.

• The largest barriers will be faculty time and convincing faculty that time they commit to assessment will be well invested. It will be a challenge to convince faculty that their role in assessment is critical and that assessment is a valuable component of faculty development. We need ideas and strategies.
• As on most campuses, our faculty believe there is plenty to do without being asked to determine if all our students have achieved a certain QR level, and in addition they do not know how to do that.
• Faculty of our professional schools know the value of assessing student learning and are willing to do the work because they are very motivated to ensure their students are competent in their field. Our arts and sciences faculty share that motivation about student success, but are less enthusiastic in taking the time to do the assessment.
• The challenge is to integrate what needs to be done with assessing QR with the other broad assessment initiatives that are now going on here. We are in the process of trying to develop a culture of assessment, challenged by how to put our arms around the different tools and instruments and measures already in place (and emerging) in individual departments, while avoiding simple check lists-in order to keeping our assessment standards appropriately high, while ensuring ease of administration and to be certain that what we are doing truly enhances the learning of our students.
• The main challenge we face is the lack of consensus about what assessment is and why it is important for our community. Many of the recent discussions among faculty reveal concerns about a lack of clarity in what is being assessed. In addition, there are concerns about the ability to make meaningful conclusions when assessing courses from disparate disciplines. But, since we've just completed a curricular revision, it is the perfect time to implement a meaningful assessment system...and we are beginning to prepare for an accreditation visit in two years.
• Our assessment tools are out-of-date (we've used the same tools for over ten years) and no one knows how the information that we collect is used. The tool is not on-line, so data have to be entered by hand and it does not come back to the departments in a timely manner. With other difficulties in assessment and evaluation, we do not have much real evidence about how our students or faculty are doing.

Donna Sundre (James Madison University) responds: Despite the fact that the 'assessment movement' has been underway for over twenty years, many institutions have made no progress. Faculty shouldn't be asked to invest precious time and energy in fruitless activity. If we cannot convince one another that assessment of quantitative reasoning is important, we should not be assessing it. A commitment needs to be made to guide faculty in: 1) deciding what is important to assess and 2) identifying or crafting meaningful instruments and credible procedures (sampling and data collection processes) to assess important student learning outcomes. Faculty must be involved in all stages of an iterative assessment cycle that is guided by their questions. This requires leadership, attention span and resources, but if we do it well, it will be among the most intellectually stimulating and scholarly exercises we engage in. We will improve student growth and development; this is the primary reason we elected a career in academe.

• Our faculty are resistant to interdisciplinary work and thus an effort to assess our emerging QR program needs to address that resistance.
• We are currently evaluating the general education curriculum. If the model remains more distributive in nature, the integration of QR will be left to the individual instructor and this will be a missed opportunity to connect QR with other disciplines.
• [One] difficulty will be in getting our senior administrators on board. We have no common requirement, such as a sophomore portfolio, that crosses all departments and that could serve as a home for QR.

Caren Diefenderfer (Hollins University) responds:

• The challenges that we have in integrating QR and assessment of QR across campus is that our faculty are fatigued with change (new general education program, new writing initiative, revisions of majors, etc.).
• Our campus is in the midst of significant change, and trying to integrate QR into the curriculum at this time is not easy.
• The challenge is that we've gone through a process of institutional 'transformative change' over the past ten years which did not include QR, which was only mentioned but not given specific resources to move toward a new QR initiative. We will have to figure out how to convince faculty that expending energy to assess the achievement of QR goals and learning outcomes is something that is important to do in addition to everything else they are always doing-and then find the resources to make it happen.

Jeanne Narum (PKAL) responds:

• The time it takes for approval through faculty committees to get any major change discussed and approved is likely to delay implementation of our dream of establishing QR on our campus. Luckily, we can weave this effort into the current initiative to improve general education (leveraging both efforts).
• Outside our department, it will likely be inertial forces that make it difficult to integrate QR throughout the curriculum, rather than merely isolating it in the Math Department. That said, the Gen Ed committee is supportive of QR and [the integration of] basic QR components into the general education. Our new learning and teaching center is also supportive of interdisciplinary initiatives such as this, so there is some hope.

David Bressoud (Macalester College) responds: The tendency to keep doing what has been done in the way it has been done is powerful, and with good reason. It takes energy to change. The key to selling new programs with an institution-wide impact is to build a cohort of allies, faculty in a variety of disciplines who share your vision and who have been involved in shaping it, so that they have a stake in bringing it to fruition. Change requires a combination of patience in laying the groundwork and the ability to marshall allies and move quickly when an opportune moment-such as revision of the general education requirements-opens up. I describe such a process in my Numeracy article, "Establishing the Quantitative Thinking Program at Macalester," .

• Since we are such a large institution, many of the natural courses in which QR would be addressed are taught by part-time or full-time non-tenure track faculty. Due to the turnover in these important positions, it is difficult to provide the right kind of ongoing faculty development within the Math Department.
• We need to garner resources to do faculty development in this area, as part of the larger discussions that will need to take place to make this happen.
• Another challenge is to differentiate between the math proficiency requirement and the new QR requirement.
• Our faculty, of all generations, have shown themselves very eager to participate in workshops and discussions about how we can reach and engage as many of our students as possible in new and challenging pedagogical approaches. Our problem will be moving them from supporting the QR initiative to getting them to figure out how they can teach their courses to serve the QR requirement. This is a challenge, and we will need to become better at showing the relevance of QR for courses in all disciplines, and to become clearer on how QR differs from any of the other similar requirements in science or business.
• While our faculty have had discussions about QR in the past, we have a cohort of new faculty who were not part of that discussion, and there will likely be a need for professional development for faculty without experience in developing, teaching, or assessing courses with significant QR components. The good news, however, is that the Gen Ed evaluation gives us an opportunity to get faculty to explore common issues and to be exposed to what programs like QR can be.
• Since thinking about QR is relatively new for our campus, we have little experience in doing faculty development that serves our QR goals.

Suzanne Mente (Alverno College) responds: Faculty development is an ongoing concern, whether it is because the QR program is new, the faculty member is new to the institution due to turnover, or the faculty member is new to the QR course due to teaching rotation (a group that often gets overlooked in an established program). Consider collecting examples of syllabi, learning experiences, and/or assessments from different disciplines that illustrate what QR means on your campus, and then creating a resource notebook that can be used for training new faculty. This can help with consistency and makes the job much easier. To help distinguish between math proficiency and QR, you might want to share the chapter Quantitative Literacy and Mathematics in Lynn Steen's book Achieving Quantitative Literacy: An Urgent Challenge for Higher Education.

• We have great depth in the faculty in our math, physics, and business departments with a wealth of knowledge and interest in working on QR matters. We have scientists working collaboratively with artists to provide opportunities for students in all the Fine Arts to develop their QR skills. We have nationally known scholars in statistical literacy. We have historians and journalists and even a theologian who are interested in and equipped to introduce humanities students to QR.
• Our faculty are, as good colleagues ought to be, willing to work together to improve the academic program and [to develop] methods of assessing how well that program is achieving our common goals.
• Our faculty have shown a great willingness to transcend traditional disciplinary boundaries to provide integrated and contextualized learning opportunities.
• We have had broad support from the natural and social sciences.
• A group of dedicated faculty and student affairs professionals have begun to work on revising the College of Arts and Sciences core requirements, and over 180 undergraduates have expressed interest in participating.

Len Vacher (University of Southern Florida) responds: Unquestionably many institutions represented at the workshop are well positioned to make progress toward their goals discussed amidst the fall colors at Carleton last November. It is evident too that there are numerous flavors of QR and successful approaches to QR education. The venue for documenting and disseminating case examples to the broader QR community is Numeracy (http://services.bepress.com/numeracy), the open-access, scholarly journal of the National Numeracy Network. The journal, which has just started its second year of publication, already includes three case examples documenting the development and implementation of QR programs (QR across the curriculum at Colby Sawyer College, by Ben Steele and Semra Kilic-Bahi in issue 2 of volume 1; an interdisciplinary quantitative thinking program at Macalester College, by David Bressoud in issue 1 of volume 2; and an interdisciplinary quantitative writing program at Carleton College, by Nathan Grawe and Carol Rutz in issue 2 of volume 2). The expertise and enthusiasm shown at the Carleton workshop gives promise that more will be forthcoming.

• Faculty approved legislation which values student exposure to QR courses and QR assessment.
• The BSc review and new curriculum have sparked a level of momentum and buy-in, which is unprecedented in science at the University.
• One of our general education requirements is a course that helps develop mathematical, computational, or statistical reasoning. Current courses that satisfy this requirement are in such disciplines as economics, psychology, mathematics, and physics. We will begin assessing the revised general education requirements this year.
• We are in the process of reconsidering our general education/core curriculum, and consequently we are now evaluating our current model in contrast to others. The University's Academic Affairs committee is leading this process and is currently considering a QR-type requirement. As a committee with broad participation from across the faculty, a strong endorsement from this group would further QR very significantly.
• The possible switch to semesters offers a rare opportunity to do things differently (and better).
• The present opportunity to support the QR initiative is our new general education curriculum that we are evaluating. The faculty involved in teaching these courses come from divisions across the college and the courses are interdisciplinary. The interdisciplinary nature of the courses gives faculty a chance to work together on a common content from various perspectives including QR. Therefore, some faculty who may not traditionally teach QR now have exposure to QR in the classroom. The new curriculum also provides a unique opportunity to expose all students to QR in a deliberate and systematic fashion across the 24 credits of common course work. Finally, the interdisciplinary nature of the curriculum content provides the opportunity to teach QR across many content areas beyond the natural sciences and mathematics fields.
• The General Education Council (a faculty committee that reports to the Senate) is charged with assessing general education and will develop policies and procedures for assessing the new QR requirement. This committee will be able to draw on the expertise of our Assessment Council (another faculty committee that works to help faculty comply with our outcomes assessment policy).
• Because we have just completed a curricular revision, this is a perfect time to implement a meaningful assessment system. While we have not yet reached agreement concerning how to assess, most of the faculty agree that we need to assess our general education, including QR courses.
• The General Education Committee is supportive of a more integrated QR program and is developing a new General Education program that would allow for more integration between the basic QR components of the program and the breadth requirements of the program.
• In many respects the Block Plan is an asset that allows us to implement, evaluate, and adjust initiatives up to eight times in an academic year as opposed to only twice with a traditional semester plan.
• Another asset is a Strategic Plan that is focusing one of its goals on the Integration of the Core Curriculum. This project has full institutional support and involves faculty from all our schools, not just the liberal arts disciplines that teach core courses. We are in the process presently of developing and vetting a new set of learning objectives for an integrated core.

Bernie Madison (University of Arkansas) responds: Almost all colleges that have considered undergraduate learning goals have included something akin to quantitative reasoning. Once the menu-of-disciplinary-courses curricular response model is rejected, the reasonable response is across-the-curriculum. Thinking across the curriculum pushes toward the broader and integrative (and correct!) conceptualization of QR. When everyone has a stake in QR education then interdisciplinary, synergistic, inventive, and collaborative approaches can be developed. Faculty develop a shared vision of QR, and this vision suggests assessment, albeit probably different and difficult. A productive learning community of faculty develops around QR.

• We just completed our reaccreditation work, a process that left us all with a greater sense of the need for active and continuing assessment of all programs at the University.
• The University has mobilized faculty and staff across campus for reaccreditation. The College of Literature, Science, and the Arts has committed its resources to an evaluation of all its graduation requirements for undergraduates. The initial focus will be on the general education requirements "examining each individual requirement's articulation of desired outcomes, refining the articulation as needed, and determining how best to assess the achievement of these outcomes." The long-term outcome that the College aims to achieve from this comprehensive evaluation is its own transformation to a culture committed to innovation and assessment. In such a culture, both at the College and department levels, the faculty will be in regular, intentional conversation about the curriculum and its specified learning outcomes. Assessment will be a regular and expected process integrated into curricular oversight and change.
• Reaccreditation review is coming and assessment is an important part of this review process.
• External pressure from accrediting and granting agencies moves assessment to a higher level of priority within the institution.
• The campus is preparing for a reaccreditation review, so we are focused more than usual on assessment.
• In part as a result of the recent accreditation process, there are now well-established structures within the University governance system tasked with the role of ongoing programmatic assessment, including the University's Institutional Strategy and Assessment committee.
• Because we are in the process of preparing for an accreditation visit in the fall, many see this as an opportunity and as motivation to have a plan in place within the next year.
• Our upcoming accreditation puts some external pressure on us to have a QR assessment in place.
• The campus wide awareness of the up-coming accreditation is helping to focus attention on assessment in general, including QR assessment. Several academic departments (anthropology and sociology, educational studies, and political science among others) are considering requiring methods courses with a greater emphasis on statistics. That interest may be harnessed to propel parallel initiatives in QR assessment in those departments.

Nathan Grawe (Carleton College) responds: In previous accreditation cycles, campus visit teams emphasized the assessment process. Increasingly, attention is moving toward how assessment information is being used on campus to inform decision making. While this transition raises numerous challenges and concerns, it also presents an opportunity for QR programs. Past or future reaccreditation visits can frame a conversation with administrators about the need for QR-specific professional development support to respond demonstrably to assessment results.

• Our learning and teaching center is a forum where faculty gather to discuss a variety of issues related to pedagogy and the University's educational mission. The center supports a very active agenda of well-attended faculty development and teaching workshops each semester. It has sponsored several workshops that targeted QR: one recent offering was co-taught by an electronic services librarian and a faculty member from our Mathematics, Statistics, and Computer Science Department, while another focused on teaching faculty how to use the computer to further political and social science research through data mining. We are confident that the center will continue to provide an ideal location for discussion and dissemination of ideas about QR.
• The college wishes to establish a QR center that will parallel our Writing Center, both of which will be housed within the learning and teaching center. We are at the beginning of the planning process, and have the support of the relevant faculty and the administration.
• In the last year the dean of faculty has created a learning and teaching center at the college, and one of the founding co-directors has asked me to give a QR presentation and workshop to faculty in the spring. This will be an opportunity to inform faculty about QR at the college as well as to present specific ideas and examples of how the QR program can strengthen their students' work and writing. The aegis of the newly formed center may encourage the attendance and involvement of new faculty and faculty not yet in the QR fan club.
• The University's teaching and learning center will be an asset in this process.
• The new teaching and learning center is a forum for presenting specific ideas and examples of how the QR program can strengthen the learning of their students. As the new center also focuses on new faculty, this will be a way to involve and inform new members of our community.
• The teaching and learning center is an important resource that can provide both curricular development and faculty development for the new program.
• At the end of each academic year, faculty gather for a three-day "May College," focusing on issues of pedagogical and curricular concern to the University.
• This fall, we will pilot a writing assessment program and a teaching and learning center. Our plans to assess QR are consistent with these two pilots and will add to the momentum of improving student learning and enhancing teaching overall.

Corri Taylor (Wellesley College) responds: QR initiatives and programs, ultimately in support of student achievement, often begin with faculty development, and a Learning and Teaching Center can be a tremendous asset in both endeavors. Just two weeks ago our LTC, QR Program, and Writing Program cosponsored a half-day faculty workshop on helping students "argue with data." Twenty faculty members attended and the response to the program was "More great collaborations, please!" Our own LTC has also been invaluable in training peer tutors and matching students with tutors. I wish you all wonderful collaborations with your LTCs, too!

• One mark in our favor is that our institution has been piloting different assessment approaches in different courses and departments over the past three years, so there is a growing body of experience among our faculty that can be leveraged as we move ahead with QR.
• We hope to be able to leverage the work in the freshman writing courses to both advance QR and its assessment.
• The Faculty Learning Community that we will have this academic year will help us at the ground level.
• I think the best way for the Quantitative Center to launch a successful initiative is to work to enlarge the scope of the writing initiative to include best methods of presenting quantitative information in writing assignments. I want to continue to investigate effective techniques for incorporating data and graphical information into written material and argument, with the aim of creating workshops for those faculty whose students need to be able to write cogently about data.
• The University has also recently received a major grant to assess our diversity requirement, which we envision helping us to learn how to assess our general education curriculum broadly.
• Four years ago we began a Writing Initiative Grant program in which faculty outside of English received year-long training in teaching writing (and a sizeable stipend) in return for a promise to teach three offerings of the first-year writing course. This program has been very successful; this fall, more than half of our first-year writing courses will be taught by non-English faculty, and we are well-positioned both in training and in culture to move to a curriculum that truly incorporates writing across the curriculum. Our QR initiatives are a natural extension of this writing program. While faculty will not be trained to teach introductory math classes, they will be learning to embed skills and strategies from other disciplines into their courses.
• We have had great success in implementing a new writing program and marrying faculty development and assessment in that implementation. Close to half of our faculty participated in one or more of three opportunities to look at student writing last May. While faculty were determining their values for writing, they were simultaneously learning about how they write assignments, grade papers, and provide feedback to the students. Our faculty reported that they found these workshops informative, insightful, and invigorating.
• Another asset is our culture of high quality faculty development; our faculty are accustomed to participating in institutional programs that encourage them to explore new pedagogical areas. In addition, we recently received a FIPSE grant to create sustainable faculty development programs in support of integrative learning. We are using these funds to develop programs that will help our faculty realize and sustain the goals of our new curriculum, including those relating to QR.
• There will like be some political wrangling associated with the approval of QR courses, but since we already have a well-developed assessment policy, once the requirement is in place, we do not see any significant barriers in the realm of assessment.
• Our course management is designed to archive student work electronically and link that student work to programmatically defined learning objectives.

Carol Rutz (Carleton College) responds: A thriving faculty development program can certainly be adapted to leverage QR as well as information literacy, writing, and other pedagogical strategies. Faculty like efficiency, and if they are convinced that attention to QR serves other goals, e.g., better arguments, they are more likely to pursue the techniques that will lead in those directions. Nathan Grawe and I have submitted an article to Numeracy about faculty response to integrating QR with writing across the curriculum (WAC). We welcome feedback on that article and hope it proves helpful to others.

• Support from the President and the Vice President for Academic Affairs is a huge asset.
• We have strong administrative support.
• The director of undergraduate studies position was recently moved to the Associate Provost level, and the current Associate Provost for undergraduate studies was one of the leaders in the move to the new General Education program and is a strong supporter of assessment. We have a new Dean of Science and Mathematics who is also very passionate about student learning and achievement, and he has worked hard to understand our general education program and support our work with it.
• Both the Vice Provost for Assessment and Student Learning and the Vice Provost for Faculty Development (both former A&S faculty members) will be strong supporters and are willing to participate in this effort.
• We recently established an Office of Institutional Research and Assessment thanks to a 5-year $1.3 million dollar grant from the USDOE. The Director began work in July, and we are now running a search for an Associate Director for Assessment. There will soon be a level of institutional support for assessment available to us that we have never had before.
• The provost is helping to fund this effort. He seems quite interested in quantitative literacy. There is growing interest among the faculty about assessment. We will be applying for some internal funding ($200K-$250K) this fall to establish a Center for QR.
• The College's administration has embraced these efforts by approving the establishment of a quantitative reasoning advisory committee and accepting its recommendations concerning changes to existing policy and procedures. Such support is expected to continue and grow.
• We have resources and financial support from the top-levels with the University, and the goodwill of academic staff at the course-level.
• We have recently added a second staff member in the Institutional Research office to support assessment. We have a half-time faculty member who is released to assist with assessment. The Associate Vice President for Academic Affairs has also held the position of Assistant Vice President of Assessment (a position that has been rolled into her current position). Because her background is assessment planning and implementation, she is very supportive of assessment.

Rebecca Hartzler (Seattle Central Community College) responds:There are several ways that administration can be of service with integrating quantitative reasoning across disciplines. Definitely having an understanding of the increased quantitative burden on our citizens that places increased immediacy and responsibility on the college to create effective QR programs. But also an effective administration can support the faculty in finding funding for curriculum development or for initial pilots and additionally in helping to clear barriers in implementation for example having Deans acting as advocates across the college to increase interdisciplinary participation, Student Services helping with enrollment of new courses and the Assessment/Institutional Effectiveness office assist in creating outcomes measures. This type of collaboration is necessary and often means that first and second level administration is championing the work of the faculty.