Winona State University Geoscience Program Assessment: Our Evolution
Cathy Summa, Department of Geoscience, Winona State University
The Winona State University
(WSU) Geoscience program has
been
wrestling with assessing our curriculum and
program
effectiveness for the better part of the past decade.
We began our
work in earnest in the spring of 2003,
motivated by the
arrival
of two new colleagues (representing 50%
of the faculty). As the department has grown and the curriculum
has
evolved, our assessment strategies
have likewise evolved.
WSU
enjoys
an institutional culture that embraces assessment.
The institution hosts an annual "Assessment Day" early
in
the spring
semester (mid-February)
for which all
classes except
those
with only
one weekly meeting (including laboratories) are officially canceled. The
day is
divided into a
morning session where students
are
encouraged to participate in
exams
that
assess university general-education goals and student
satisfaction.
The afternoon session is
reserved for "departmental activities." Departments
are encouraged to
use this time to bring
students together for
program
assessment activities; limited institutional funds
to support departmental progress are available
through a series of "challenge grants".
The
department felt strongly that we had to
participate in
some meaningful way so that we sent
the
message
to students that "assessment matters." Our preference
was to develop an
assessment that
measured
our
students'
abilities to approach
and resolve a field‐based
problem. Unfortunately, February in Minnesota is
a challenging time
for fieldwork,
which
forced
us to look to alternative assessments. After much
debate,
we settled
upon developing
an "assessment
exam,"
which would
be
administered to students each
year
on Assessment Day
and would be
designed
to measure
students'
progress
by assessing
the depth and
maturity of
their responses.
The central idea
was to
develop
questions that students
who completed
or were enrolled in
our introductory
sequence (physical geology and
historical geology) could demonstrate
what
they learned
in terms
of
simple rock identification or map
and cross section
interpretation, while
we anticipated that upper-level students could
synthesize
content
from
multiple
courses to provide
much
deeper
and richer analysis
in their responses.
We
made
every effort to develop
an exam
that
would challenge, but not
demoralize, our
students. This,
in itself, turned
out
to be more difficult
than we originally anticipated,
given
the disparity in
knowledge and
ability between
a new
entering student
and a student in his/her last semester.
After spending about a year developing
the exam, we
administered it
for
three
consecutive years
on Assessment Day (2005‐2007). We
made small revisions to the
exam
each year
as we
saw
how
things went;
revisions were
generally minor, and
made for clarification or to reduce
the length of the exam.
We coded the
exams so that graders had
no knowledge of
student
identity.
Grading
the exam
presented an
entire new
set
of challenges, as it was difficult to set
aside
a large
enough block of time following the Assessment
Day
to
work collaboratively
to ensure internal consistency.
Somewhat surprisingly, students were anxious
to
know
how
they
did.
Our analysis of student performance
showed that
our initial hypotheses held true:
students who
had
taken more
geoscience
courses performed
better
on the exam.
However, we
found
that student performance
at the upper level was not as strong
as we anticipated it
would be.
When
we interviewed
students (graduates)
to
understand why, we
learned that senior students
didn't take
the
exam
seriously.
They
completed
it quickly
and
gave perfunctory answers,
resulting in lower
scores
than
expected. In
short, our
data
analysis revealed that we
needed
to find
ways to incorporate our
assessment strategies
into the fabric of
our
curriculum.
As
a result of our experiences,
we have since abandoned our departmental "assessment
day"
activities, and
instead
use
that time for
faculty planning
toward on‐going assessment.
We
have
opted
to
move
forward with
two parallel strategies,
both just
in their infancy.
We have agreed
to create a
set of exam questions that will be
incorporated
into
the final
exam of core courses,
beginning with
our
introductory courses. Student responses
to these questions will
be
tracked
longitudinally to determine if we
see
improvements in the
maturity of
the response.
Secondly, we have decided
to implement
a student‐learning portfolio into
our curriculum.
We are
piloting
the
portfolio
project in two classes
this
spring
semester
(in
Earth
and Life through
Time,
the
second
course in
the
major sequence; and in Sedimentology and
Stratigraphy, a junior-senior
level
writing intensive course).
Truth
be told, this is a lot of work.
It's
difficult to hold
the
creative
tension
needed
to
work
through
the development process and not fall back on collecting numbers, whether
these
be
grades
earned in specific
courses
or
numbers of students
going
on
to graduate
programs.
It's
challenging
to
think
through developing
exam
questions
that will elucidate connections
through
the curriculum and
it's
equally
challenging
to think through how
to scaffold
student
development
through
portfolio creation and maintenance.
Fortunately, our
faculty are all
committed
to assessment as a means of both demonstrating our successes
and
improving
upon
our weaknesses.
The process
is fascinating
and
we're all learning
a good
deal.
It's helped
us,
collectively, really come to true consensus
on
what
we believe
to be
the most
important
aspects of our
program
and
on our hopes and
dreams for our
graduates.
Perhaps
most significantly, our assessment process has affirmed
that we
agree on ~98% of
what we
do,
making
it
much easier to resolve
conflicts when
and if they arise.
Stay tuned.