Winona State University Geoscience Program Assessment: Our Evolution
Cathy Summa, Department of Geoscience, Winona State University
The Winona State University (WSU) Geoscience program has been wrestling with assessing our curriculum and program effectiveness for the better part of the past decade. We began our work in earnest in the spring of 2003, motivated by the arrival of two new colleagues (representing 50% of the faculty). As the department has grown and the curriculum has evolved, our assessment strategies have likewise evolved.
WSU enjoys an institutional culture that embraces assessment. The institution hosts an annual "Assessment Day" early in the spring semester (mid-February) for which all classes except those with only one weekly meeting (including laboratories) are officially canceled. The day is divided into a morning session where students are encouraged to participate in exams that assess university general-education goals and student satisfaction. The afternoon session is reserved for "departmental activities." Departments are encouraged to use this time to bring students together for program assessment activities; limited institutional funds to support departmental progress are available through a series of "challenge grants".
The department felt strongly that we had to participate in some meaningful way so that we sent the message to students that "assessment matters." Our preference was to develop an assessment that measured our students' abilities to approach and resolve a field‐based problem. Unfortunately, February in Minnesota is a challenging time for fieldwork, which forced us to look to alternative assessments. After much debate, we settled upon developing an "assessment exam," which would be administered to students each year on Assessment Day and would be designed to measure students' progress by assessing the depth and maturity of their responses. The central idea was to develop questions that students who completed or were enrolled in our introductory sequence (physical geology and historical geology) could demonstrate what they learned in terms of simple rock identification or map and cross section interpretation, while we anticipated that upper-level students could synthesize content from multiple courses to provide much deeper and richer analysis in their responses. We made every effort to develop an exam that would challenge, but not demoralize, our students. This, in itself, turned out to be more difficult than we originally anticipated, given the disparity in knowledge and ability between a new entering student and a student in his/her last semester.
After spending about a year developing the exam, we administered it for three consecutive years on Assessment Day (2005‐2007). We made small revisions to the exam each year as we saw how things went; revisions were generally minor, and made for clarification or to reduce the length of the exam. We coded the exams so that graders had no knowledge of student identity. Grading the exam presented an entire new set of challenges, as it was difficult to set aside a large enough block of time following the Assessment Day to work collaboratively to ensure internal consistency. Somewhat surprisingly, students were anxious to know how they did.
Our analysis of student performance showed that our initial hypotheses held true: students who had taken more geoscience courses performed better on the exam. However, we found that student performance at the upper level was not as strong as we anticipated it would be. When we interviewed students (graduates) to understand why, we learned that senior students didn't take the exam seriously. They completed it quickly and gave perfunctory answers, resulting in lower scores than expected. In short, our data analysis revealed that we needed to find ways to incorporate our assessment strategies into the fabric of our curriculum.
As a result of our experiences, we have since abandoned our departmental "assessment day" activities, and instead use that time for faculty planning toward on‐going assessment. We have opted to move forward with two parallel strategies, both just in their infancy. We have agreed to create a set of exam questions that will be incorporated into the final exam of core courses, beginning with our introductory courses. Student responses to these questions will be tracked longitudinally to determine if we see improvements in the maturity of the response. Secondly, we have decided to implement a student‐learning portfolio into our curriculum. We are piloting the portfolio project in two classes this spring semester (in Earth and Life through Time, the second course in the major sequence; and in Sedimentology and Stratigraphy, a junior-senior level writing intensive course).
Truth be told, this is a lot of work. It's difficult to hold the creative tension needed to work through the development process and not fall back on collecting numbers, whether these be grades earned in specific courses or numbers of students going on to graduate programs. It's challenging to think through developing exam questions that will elucidate connections through the curriculum and it's equally challenging to think through how to scaffold student development through portfolio creation and maintenance. Fortunately, our faculty are all committed to assessment as a means of both demonstrating our successes and improving upon our weaknesses. The process is fascinating and we're all learning a good deal. It's helped us, collectively, really come to true consensus on what we believe to be the most important aspects of our program and on our hopes and dreams for our graduates. Perhaps most significantly, our assessment process has affirmed that we agree on ~98% of what we do, making it much easier to resolve conflicts when and if they arise.