Initial Publication Date: April 21, 2008

Michael Kelly

Department of Geology, Northern Arizona University, Flagstaff, Arizona

My background in geology and geoscience education

I have degrees in geology from UC Santa Cruz and Northern Arizona University and worked on some spectacular geologic framework projects in the Pacific Northwest and California's Mojave Desert, while at the US Geological Survey. Eventually I left for graduate school and then landed as geologist working on groundwater and soil contamination projects in Oregon and Washington, where I supervised geologists and engineers at the beginning of their careers. In the early 1990's I became interested in visualizations in games. At work I recognized that these new geology graduates I hired had little practical environmental experience. These people needed a place to safely make and learn from terrible field mistakes so that they wouldn't destroy our clients or the environment! One dark and stormy night I played the game "Myst" from beginning to end and decided to find out more about simulation programming.

In 1996 I was fortunate to be funded by NSF to start a project with colleagues at NAU's Department of Environmental Sciences and Dine' Tribal College to explore the use of data-rich environments in environmental geology. This led to three interactive simulations on the topics of groundwater contamination, nuclear waste disposal site selection and the use of coal as an energy source, and an associated self-contained virtual library of critical information. Virtual Reality Excursions was surprisingly long-lived (for software) but I officially retired it this semester (Spring, 2008). This simulation set was designed for 1-4 students sharing tasks on a single computer.

As a faculty member in Environmental Sciences I also developed a suite of data-rich nursing simulations centered around complex and time-dependent data for patients located on various floors of a "virtual hospital". Again thanks to NSF for the funding. Nursing students are taught to be critical thinkers before all else and this suite includes self-assessment tools designed for a single user. Also during this time I helped manage the adoption of my University's first online course management tool (CMT's): Blackboard. Across many schools CMT's have quickly become entrenched and are somewhat restrictive as to the types of ancillary media and interactions allowed.

I worked with Nancy Riggs on an Introductory Field Methods class that explored changes in student understanding after their interactions with a virtual environment. We had several years of data on student performance in a learning and assessment system that had held the instructor, content and rubrics constant. Then for two years we added student-driven exploration of the geologic field area using a stereo geowall and observed improved student scoring. see JGE article. Working on this well-controlled investigation allowed me to look back with new understanding at the several decades of educational literature that concluded that there was no difference in student performance between multimedia-based educational materials and standard educational materials. In my opinion the change occurred when we used an immersive and somewhat collaborative environment as the multimedia intervention.

Over the last several years I have been working with a team of geologists and visualization experts developing a physical geology textbook. We are discussing how to get the rich, 3d visualizations we have produced for the book into a multi-user, assessable framework. If you have acrobat reader 8 and windows (sorry! sorry!) these are a few examples of some of our object-based 3d visualizations using the cool but limited Adobe 3D engine:

Silica Octahedron
SAFOD Facility

Some Questions I have......

I think it is going to important to find ways to assess individual learning in multiplayer online gaming. I see many faculty leaning heavily (and with innovation) on course management tools to perform automation of assessment, and I think this will only catch on.

How then can we bring these assessment tools into rich multiuser game environments?
How can we value team learning in online virtual environments, has anyone seen this discussed in the literature?

In the online worlds of Second Life and World of Warcraft, the terrain texturing methods are not good for displaying detailed surface geology. High resolution texturing is bandwidth intensive and difficult for average graphic cards to render.

How can we simulate the surface of a detailed geologic area ?

Michael Kelly --Discussion

Mel Huff
Apr, 2008

Ok, I know we're supposed to be commenting on the stuff on the webpage, but I have to say that I LOVE your new geology textbook!!! I was on a conference call with Reynolds when it first came out. I am adopting it for the fall and am super excited. I love the way it's organized!! I haven't had time to look at the CD/DVD that came with it, but are there some of these games and simulations on it?

Mel

826:2425

Frank Wattenberg
Apr, 2008

One particularly promising possibility with games and simulation is the ability to do richer kinds of assessment. Ben Schneiderman and others at the University of Maryland have done some particularly interesting work in this regard. They collect "learning histories" as students use a simulation. Their work is in engineering and they are able, for example, to see how a student diagnosis and attempts to repair a simulated problem. Feedback is based not just on the right answer but how that answer was reached -- for example, what measurements did a student make before changing a part.

826:2432

Sean Fox
Apr, 2008

Is this sort of formative assessment/feedback then integrated into the game? One of the interesting strengths of many games is that there's some sort of explicit score-keeping. So you always know how well you're doing at any point. Formative assessment takes a similar (if not identical) role in learning. Should 'winning' the game be engineered to be identical to 'reaching the learning goal'?

Is that even possible in situations where the learning goal is hard to measure (especially in an automated way)?

826:2437

Mike Kelly
Apr, 2008

Thanks Mel, We are working on the 2nd edition and improving it! Want to pilot the Environmental Science book? Yes there are some nice simulations on the DVD and I have used them as games and assessment fodder myself. We should talk offline!

Frank - I totally agree about looking deeper into the process of student decision making. If we could assess more of the steps not only could there be intervention and correction but also reinforcement of new interesting and valid pathways, giving credit (better grades) even if the "answer" was off due to calculation. I think students would buy into this as well.

826:2438

Cathy Manduca
Apr, 2008

Hi Mike- on the assessment front -- you mention that you had self-assessment built into the nursing materials -- presumably self assessing their critical thinking? can you say more about this?

On a separate front:

'The change occurred when we used an immersive and somewhat collaborative environment as the multimedia intervention.'

So those are two very different things/effects in my mind. Do you think both are important, one is dominant? Why do you think they are important? Can you say more-- or should I read the paper more completely?

826:2440

Sam Swanson
Apr, 2008

Hi Mike --

I like what you are doing. I too am interested in the nuts-and-bolts issue of assessment. Some way to track the way a student approaches a problem/game is as much an issue for me as the 'right' answer.

Sam

826:2451

Mike Kelly
Apr, 2008

Hey Cathy - I built an assessment module that models a national competence exam required for a R.N. license. So nursing knowledge and skills associated with safe and effective practice were tested. In terms of critical thinking, many of the questions would pose multiple preexisting conditions (not combined previously) as well as current vital signs and meds, and then ask for a correct diagnoses and plan.

The students in the study published in JGE reported that the stereo virtual world representation of SP Crater gave them survey knowledge (see Montello) of the field area, which I think was the major change. The collaborative nature had an effect of forcing them to complete the exercise in the time constraints - many students reported afterward that they wanted to use the geowall by themselves!

826:2457

Alan Levine
Apr, 2008

Bring back VR Excursions! That was way ahead of it's time.

I'm "game" for something where students document their game/sim experience as say blog, narrative, media (audio, video) format, as something of a personal assessment.

826:2467

Mike Kelly
Apr, 2008

Alan,
Great idea about the student documented personal assessment but it could be harsh! Have you read some of those reviews on "rate my professor"?

826:2475

John Knox
Apr, 2008

Mike,

My intro meteorology textbook co-author Steve Ackerman (at UW-Madison) and I are looking to go very virtual with the next edition of our textbook. So far, partly due to publisher pressure (our publisher is moving into the 20th century shortly), our electronic applets have not been as fully integrated into the overall content as we'd like. With a new edition and new publisher, we intend to do much more. Would like to be in touch with you about how to do that, since you've done it in physical geology.

Back to your question about team learning, the online weather forecasting contest I mention on my own page does do team as well as individual scoring methods. Since different teams may have different numbers of participants (minimum 5), the team score is a combo of the top 5 scores, the median five scores, and all team members' scores. This is one small step toward encouraging team learning, since it attempts to measure the aggregate of multiple individuals' learning and rewards participation by all. As minimal as it sounds, this is affecting curriculum--to improve our team's score, I'm looking into creating a 1-credit course for all team participants next fall that will encourage/require daily participation in a group forecast discussion.

826:2476