Develop an Assessment Plan

Assessment is just collecting program data for a specific purpose. Just as an effective research program grows from a carefully thought out research plan, an effective assessment process begins with effective planning. Developing a program matrix is a powerful tool for creating and implementing an assessment plan that looks at program-level learning goals at the department scale. This kind of planning involves ways of gathering formative as well as summative data.

Jump Down To: Examples | References and Resources

Formative Assessment

The program matrix can illustrate a wealth of information that can be used internally to provide clarity and course corrections for departmental programs. Formative perspectives include:

Overall Learning Experience
Does the existing curricular structure combined with co-curricular offerings provide sufficient opportunities to develop the ideal student you want to produce?

Sequences
Pick a particular theme from your department's program-level learning outcomes (e.g. geologic time, research skills, or verbal communication). Where, how, and to what extent is this actually addressed across the program? Reflecting on the variety of ways in which students move through the program(s), do the offerings give appropriate scaffolding and practice that support students achieving mastery? At each level of the program, are the skills and concepts presented appropriate for the students' learning abilities? Do students have regular access to the courses that emphasize particular outcomes at appropriate stages in meeting their degree requirements?

Pedagogy 
What courses routinely offer field exercises, problem-based learning, computer modeling, writing or oral presentations, application of quantitative or statistical methods, etc.? Is a faculty member particularly successful with a specific teaching method that might be more widely useful?

Affirmation
Every department makes claims about the things that it does well: field experiences, independent student research, GIS training, etc. The program/departmental matrix can be used to back up these claims with information on where and how often these "local specialties" are addressed in the program(s).

Gap Analysis
Given the current state of geoscience research and employment, what concepts, skills, and courses are missing from your program(s)? Thinking about the trends for the future of geoscience, what knowledge, skills, and attributes will your students need to be successful going forward? Are there new courses that need to be added to meet the program-level learning goals you've set? Or, conversely, are there courses that need to be combined, redesigned, or eliminated to free up space in the curriculum?

Faculty Contributions
The program matrix is an opportunity for reflection on what individual faculty do well and how the all the faculty can work together to address the department's goals. Identify the strengths of the individual faculty that contribute to the program. In aggregate, the sum of the contributions of all the faculty should the holistic needs of all students and all of the department's program-level learning outcomes.

Multiple Degree Programs
Many departments offer multiple programs of study. For each option, does the program all or most of the critical programmatic learning outcomes? Do the learning sequences across the suite of degree options successfully address all of the outcomes?

Summative Assessment

Departmental Role and Scope documents often establish goals that must be assessed for overall departmental productivity. A matrix that looks at all the programs in a department provides ready evidence of where and to what extent stated departmental goals are being met. Each department will have its own claims, its own high-level goals. The department-wide matrix will help provide the evidence that supports these claims.Examples of goals that lend themselves to this kind of analysis include:

The department contributes to the institutional mission.
Many institutions have a "core curriculum" (i.e. distribution requirements). Specific courses can be readily identified as meeting these institutional requirements.

The department is moving towards an Earth System approach.
If this is the case, then most courses should provide a breadth of coverage of topics that show the relationships among the components of the Earth system.

The department has an integrated curriculum focused on developing higher order thinking skills.
Degree programs are designed with learning sequences that require and foster increasingly sophisticated problem solving as students progress.

The department provides authentic research experiences for students.
Courses that feature strong in-class research can be highlighted as well as co-curricular opportunities for other authentic research experiences.

The department prepares students well for their next steps. 
Whether it means transfer, go to graduate school, or enter the workforce, students receive the preparation they need to be successful. Use the matrix to demonstrate where different kinds of skills are emphasized.

The department serves society by addressing the "grand challenges" of living on Earth.
Courses that emphasize societal issues such as geohazards, natural resources, sustainability, or environmental justice are readily identified.

Examples

Measuring the Impact of Our Programs on Students

Just as we need to assess our teaching to know what our students are learning, we need to assess our programs to know whether our programs are achieving our goals. This presentation, by Cathy Manduca and Ellen Iverson, was given at the January 2007 workshop on the Role of Departments in Preparing Future Geoscience Professionals. It demonstrates, via example, how you can apply your well-honed geoscience research skills to the process of program assessment.

Geoscience Program Assessment Planning Documents

We have collections of mission or vision statements, student learning goals or outcomes statements, and other program assessment planning documents, including institutional guidelines for reviews, assessment plans, and more, contributed by a broad spectrum of geoscience departments. While each department is unique and you will need to develop your own individualized plans, this collection may help you and your department to generate ideas.

References and Resources

• The Colorado School of Mines Assessment Resource Page includes a generalizable assessment matrix designed to help engineering faculty develop comprehensive program assessment plans for their departments. There are links to completed assessment matrices from various departments on the main page.
• Instructional Assessment Resources: Program Evaluation: The University of Texas at Austin has a website of Instructional Assessment Resources, including a comprehensive set of pages on evaluating programs. From planning your evaluation through collecting and reporting your data, this site offers step-by-step suggestions, worksheets, and examples.
• Schneider, Give Students a Compass: Can General Education Rise to the Challenge?
  This posting, from the Tomorrow's Professor Mailing List, describes the American Association of Colleges and Universities' "Give Students a Compass" project. This project is an experiment in mapping expected student learning outcomes, purposefully deploying "high-impact" educational practices that help students achieve the intended outcomes, and adopting educationally meaningful assessment strategies for general education. Making general education "work" for underserved students is a strong and sustained focus of the Compass project.
• Wergin, Jon F., 2002. Departments that Work: Building and Sustaining Cultures of Excellence in Academic Programs
  This book focuses on how academic programs can make evaluation more useful and critical reflection more likely.