Initial Publication Date: May 1, 2012

SGT Activity Review Process

Background

The geoscience community has repeatedly asked for access to high-quality, peer-reviewed instructional resources, and we are reviewing assignments and activities in On the Cutting Edge online collections to help meet this need. NSF is also interested in our review process as a model for other education projects.

The review process will identify and tag exemplary teaching activities in the collection, as well as to provide suggestions to authors to help improve other activities so they, too, can be tagged as exemplary.
Reviewers will use an online review process to score the following: a) scientific accuracy, b) alignment of goals, activity, and assessment, c) pedagogical effectiveness, d) usability, and e) completeness of the accompanying Activity Sheet. All reviewers will use the same scoring rubric (Microsoft Word 55kB May1 12). Our goal is to have every activity in the collection receive two reviews.
Based on the review of the mineralogy, petrology, and geochemistry collections last summer, we anticipate that somewhere between 10% and 20% of those activities that we will review in the SGT collection will likely be ranked as Exemplary without revision.

Review process

The SGT activity review process will take place in two stages:

Individual reviews: we will assign each member of the two review teams (structural geology and geophysics) a set of activities to review. Individual reviews must be completed before the review team meetings.
Review team meetings: the structural geology team will meet face-to-face at the Williams Forum on June 12 to finish the review process. The geophysics team will meet in virtual sessions spread out over June 27 and 28.

General procedure for individual reviews:

Each activity has an ActivitySheet that was automatically generated when the author made the submission to the collection. This sheet is what a prospective user sees first, and the sheet must provide enough information to enable the user to decide whether to proceed.
Most activities will also have materials to download and review, and the download links appear at the bottom of the ActivitySheet. Some short activities, however, may be completely described on the ActivitySheets.
You will score each activity in five categories (scientific accuracy, alignment of goals/activity/assessment, pedagogic effectiveness, robustness, and completeness of the ActivitySheet) using our scoring rubric (Microsoft Word 55kB May1 12). In order be included in the Exemplary collection, an activity must have been rated 1) Exemplary or Very Good in all categories and 2) Exemplary in at least three of the five review categories.
You will write specific advice to the author about changes needed to raise an activity to "Exemplary" status if it does not rise to that level in your review.

To help calibrate your expectations, you may find it useful to browse some of the assignments/activities in the Cutting Edge collection that were tagged as exemplary during the review of the petrology, mineralogy, and geochemistry activities last year. You'll see a variety of different kinds of activities (field, lab, classroom, back-of-the-envelope calculations).

All of the reviews will be done online using the review tool developed by Sean Fox at SERC.

We will assign you a set of activities, and you will access them via your SERC account. Average time for a review has been about half an hour per activity.
Click on this link to access the REVIEW TOOL.
Once you've accessed the review tool, open the link to whatever resource you choose to review. The ActivitySheet will open in a new window. The ActivitySheet contains the description of the activity and contextual information entered by the author at the time of submission. At the bottom of the ActivitySheet, you will find links to download all related documents and files that you will review.
Please use the scoring rubric (Microsoft Word 55kB May1 12) as a guide for each category in the review, so that reviews are consistent across reviewers and across Cutting Edge. Please write your final summary advice in the same manner that you would like to receive comments. Authors have voluntarily and generously submitted their activities for community access, and helpful, constructive advice will help authors improve their submissions. When you are done with your review, be sure to click Submit in the Review Tool to submit your review.
Some activities have been previously reviewed by other projects. That's OK! You can use these earlier reviews to make your own review easier. We do, however, need to have reviews completed using our online review tool so that activities are properly tagged for search and discovery functions in Cutting Edge collections.
Conflicts of interest: please let us know if we have inadvertently assigned you one of your own activities, or if there are other conflicts of interest that were not apparent to us. Also, let us know if you are having any problems accessing the review tool or conducting the review itself.

What happens after you submit your review

Each activity will receive two peer reviews before the team meeting. During the review team meetings, we will discuss conflicting reviews and write the final summary advice for activities that need revision to achieve exemplary status.
The Managing Editor will then send personal letters to authors to: a) inform authors that their contributions have been recognized in our "Exemplary" collections or b) forward your recommendations to the authors with encouragement to make revisions to elevate their activities to Exemplary status.
At the end of the review process across the Cutting Edge collections, we will have realized the long-term goal of NSF and the geoscience community of having a peer-reviewed collection of teaching activities!