How we Assess our Students, our Curriculum, and Ourselves

Arlo Weil , Department of Geology, Bryn Mawr College

Pedagogical and Department Curriculum Assessment

The geology faculty at Bryn Mawr spends a lot of time as individuals and as a group assessing the ways in which we teach our courses (to both majors and non-majors), as well as the ways in which we develop and modify our overall curricular offerings. The present makeup of the department faculty is young (soon to be three assistant-level, and two associate-level professors) – and as a result we have made a concerted effort to develop a unifying philosophy about our curriculum that is new, exciting to our students, and importantly, pliable. This movement towards community consensus started about six years ago after an intensive external departmental review. This review was extremely helpful in providing our junior faculty at the time (including myself) a framework by which to think about and change the "established classic" curriculum. Many of the suggested changes were seriously debated in faculty meetings over the following year – and subsequently, many of the ideas were incorporated into the curriculum we presently provide (e.g., moving from a two semester mineralogy, optics requirement – to a single semester of mineralogy and a second semester of modern petrotectonics). We strongly believe that these changes in curriculum, both at the major level and at the general education level have had a significant impact on our major numbers (up to an average of 12-15 graduating students a year from an average of 3-6).

Our department has also established – at the request of the current Provost – a long-term plan, which includes a working Mission Statement and a working list of goals. This document was conceived by the chair, but was thoroughly discussed, debated and modified by the entire department.

The department has also begun on an annual end of summer two-day retreat, which is intended as a forum for discussing an array of topics that include, but are not limited to:

1. A re-examination and reevaluation of the department Mission Statement
2. The department as a community
3. Our future vision and place in the college
4. Our curriculum
5. The "Geology Major Experience" at Bryn Mawr College
6. Budget & endowed funds
7. Research
8. Concerns, desires, goals, wish lists

The purpose of these retreats is to provide a venue for faculty to flesh-out ideas, build consensus, develop solutions, and foster collegiality. Discussions are not intended to seem like a typical faculty meeting where information is conveyed and there is little discussion. The purpose is more to tap the collective insight, creativity, and energy of the faculty as colleagues.

These retreats have been very insightful and have helped to develop collegiality as well as consensus on what it is the department is trying to teach our students, and importantly how we determine that what we are doing, and what we are providing, is successful.

One of our ongoing projects that stemmed from a recent retreat is the creation of a working document of 'unifying Earth science concepts' that we feel are vital that every student understand before they leave Bryn Mawr with a degree in Geology. At the moment this is a work in progress, but we hope to have such a document in circulation by the beginning of the next academic year.

Another important goal of our faculty is to acquire, and update, the skills and expertise in pedagogy required to be a successful educator. Whenever possible our faculty have tried to participate in external pedagogical workshops – including several that have been sponsored by SERC (e.g., teaching structural geology in the 21st century) and several that have been established as part of the Bi-College community for younger faculty.

This past year (2007-2008) one of our faculty members pursued his interest in pedagogy and interdisciplinary work outside of the sciences by participating in a semester-long junior faculty pedagogy workshop supported by Bryn Mawr's Teaching and Learning Initiative (TLI). Together with colleagues in Cities, English, and History, there were weekly meetings established during the Fall 2007 semester to address the goals, needs, and worries of junior faculty. Additionally, the faulty attended each other's courses and provided classroom critiques. During that semester, new pedagogical methods were learned and old ones improved without the added stress or judgment of senior colleagues. In the Spring of 2008, a 45-page document was drafted based on observations and experiences, which will be the basis for future publication, and which was shared amongst the faculty in the Geology department.

Course Assessment

Introductory level courses at Bryn Mawr are filled with large numbers of students (typically about 50), a great majority of who are trying to meet laboratory/science requirements. These courses also happen to be the department's bread-and-butter in terms of Geology major recruitment. Our main form of assessment in these classes is the use of written evaluations – a practice the Geology department started decades before the institution required course evaluations. Several of these classes continue to use more detailed questionnaires than the generic College evaluation. Importantly, these supplementary assessment documents are quantifiable and can be tracked over time. As a department we are leaning more heavily towards instituting these types of questionnaires into all of our intro and general education classes – including documents for individual laboratory experiences to assess whether students 'get' the main intended points of a given exercise, and equally important if the enjoyed the experience and found it informative.

The historical record of these documents has taught us that:

1. Adopting the latest textbook should be an option—we should not rely so heavily on a text that the students couldn't take the course without it. Differences between editions are usually trivial—often even the page numbers are the same! Consequently, in some of these classes we have made a text optional, which has never resulted in a single complaint.
2. Students appreciate being given original readings—like selections from Darwin, Wallace, etc. (ones that they can understand), or articles from the primary literature (Science, Nature, Journal of....) rather than a text book summary of someone else's' work accompanied by a photograph of the author.
3. It is clear that students want to learn from the instructor, not from a textbook that we interpret or explain to them—the use of relevant anecdotes, examples, humor, etc. from our own experiences mean a lot more to them than textbook insets, web-based cartoons, videos, etc.

An important component of all of our introductory courses is the introduction of students to Geology through field trips. Here, evaluations have played a very influential role in determining the nature of field trips and there practices. An example being - our annual 100-level 3-day trip was cut to two tightly packed days that are focused on individual courses. At the same time, although some students complained about the time away from campus, the introductory field trips always receive high ratings on our evaluations (typically >4.0/5.0), and more often than not are regarded as the best part of the course, and led to many major declarations.

Intermediate and advanced level courses in Geology at Bryn Mawr generally involve between 8 and 15 students (often ~10) and evaluations are not a numerically rigorous way of getting assessment, unless there are clear trend. (An exception is our larger 200-level course like Evolution 236 [limited enrollment ~43] and Natural Hazards [often ~50 students], where evaluations early on revealed that a team taught approach, with two faculty present at every class meeting provides continuity, supplemented by visiting specialists, was very much appreciated by the students.)

From 200-level evaluations, we have learned that having students give evaluated oral presentations is much appreciated. Some students looked shocked when told they were going to have to make an oral presentation (and some avoided the task by missing class on the day they were due), but many others asked for more such opportunities. It is clear that our emphasis on oral presentations at the 200-level has made our oral senior thesis presentations much better over the past several years.

Some of our core 200-level courses require a semester-long research project (e.g., Palobiology 203 and Structural Geology 204), which involve a field trip to secure research material. The field trips are a strong bonding experience for students (many of whom did not know each other) and for the faculty/students—(Virtually all geol alums still reminisce about Geology field trips!). Instead of just grading these research papers, we try to give as much feedback to the students as possible, by editing them, returning them for revision, and requiring vigorous journal-level standards for rewriting of a technical, scientific paper. Feedback from evaluations and from discussions with alums indicates that this has been one of the longest-lasting experiences from some of our courses.

We also use oral examinations as a way of assessing student understanding – similar to oral examination assessment in graduate school. These exams allow our faculty to test the student's ability for critical thinking, as well as provide a means of evaluating how affective we are at conveying the important points in a course to the students. This is vital, because our core courses are all gateway experiences for the geology major and we want to ensure that they will succeed in their future courses that are built on a foundation of mineralogy, sedimentology, paleontology, and structural geology.

Assessment in 300-level courses is more difficult, and probably can only be accurately obtained from alums. Popularity of a course does not mean rigor, and difficulty is often inversely proportional to popularity. Retrospective assessment from alums, once they have been exposed to graduate school or professions, has been very influential in guiding our course revisions. If there is a central theme it is --- require more writing, oral presentations, and independent research.

The senior research thesis in Geology has always been the capstone experience for our majors. It requires a one- or two semester research project developed with a formal thesis proposal, an oral presentation before the entire department, and it is read not only by the advisor, but by the entire faculty, and a grade is determined by faculty consensus.

In preparation we mentor students on projects that interest them, and discuss with majors what subjects they may wish to study in advance of their senior year. They are assisted during the year by participating in a senior thesis writing seminar, where they discuss ideas with their peers and faculty.

Assessment of this experience is almost exclusively by alum feedback. Consistently, over the past four decades, alums have regarded the senior thesis experience as the most worthwhile experience of their Geology major at Bryn Mawr. That said, the recent trend toward increased numbers of majors and options (not only Geology, but also Environmental Studies and Geoarchaeology) are making this labor-intensive experience difficult to continue at the same level. Modification of our senior thesis program is in order, and the department is actively and creatively re-imaging the senior experience so as to benefit everyone involved – both students and faculty. With increasing numbers of geology majors, this type of senior experience is not feasible due to:

• Faculty time/projects to sustain this experience for all students
• Money associated with each research project
• Set system for evaluation for fair and balanced evaluation of all students