HSG Activity Review Process

Background

The geoscience community has repeatedly asked for access to high-quality, peer-reviewed instructional resources, and we are reviewing assignments and activities in On the Cutting Edge online collections to help meet this need. NSF is also interested in our review process as a model for other education projects.

• The review process will identify and tag exemplary teaching activities in the collection, as well as to provide suggestions to authors to help improve other activities so they, too, can be tagged as exemplary.
• Reviewers will use an online review process to score the following: a) scientific accuracy, b) alignment of goals, activity, and assessment, c) pedagogical effectiveness, d) usability, and e) completeness of the accompanying Activity Sheet. All reviewers will use the same scoring rubric (Microsoft Word 55kB Apr20 13). Our goal is to have every activity in the collection receive two reviews.

Review process

The HSG activity review process will take place in two stages:

• Individual reviews: we will assign each member of the review team a set of activities to review. Individual reviews must be completed before the review team meeting in Albuquerque. There will be a webinar for reviewers Thursday, April 25 from 8-9 EDT.
  

• Review team meeting: the review team will meet on Tuesday, June 4 in Albuquerque starting at 8:30 am (this is the day before the optional workshop field trip).

General procedure for individual reviews (you can also download the PowerPoint slides (PowerPoint 2007 (.pptx) 2.6MB Apr26 13) from the Review Team webinar on April 28):

• Each activity has an ActivitySheet that was automatically generated when the author made the submission to the collection. This sheet is what a prospective user sees first, and the sheet must provide enough information to enable the user to decide whether to proceed.
• Most activities will also have materials to download and review, and the download links appear at the bottom of the ActivitySheet. Some short activities, however, may be completely described on the ActivitySheets.
• You will score each activity in five categories (scientific accuracy, alignment of goals/activity/assessment, pedagogic effectiveness, robustness, and completeness of the ActivitySheet) using our scoring rubric (Microsoft Word 55kB Apr20 13) . In order be included in the Exemplary collection, an activity must have been rated 1) Exemplary or Very Good in all categories and 2) Exemplary in at least three of the five review categories.
• You will write a summary evaluation of the activity.
  

To help calibrate your expectations, you may find it useful to browse some of the assignments/activities in the Cutting Edge collection that were tagged as exemplary during earlier review activites over the past two years. You'll see a variety of different kinds of activities (field, lab, classroom, back-of-the-envelope calculations).

• Schreinemaker's analysis problem #4
• Idaho field trip in physical geology class
• Heat capacity of minerals: a hands-on introduction to chemical thermodynamics
• The rock cycle in chocolate lab
• Using field lab write-ups to develop observational and critical thinking skills
• Back-of-the-envelope calculations: weight of gold
  

All of the reviews will be done online using the review tool developed by Sean Fox at SERC.

• We will assign you a set of activities, and you will access them via your SERC account. Average time for a review has been about half an hour per activity.
• Click on this link to access the REVIEW TOOL.
  
• Once you've accessed the review tool, open the link to whatever resource you choose to review. The ActivitySheet will open in a new window. The ActivitySheet contains the description of the activity and contextual information entered by the author at the time of submission. At the bottom of the ActivitySheet, you will find links to download all related documents and files that you will review.
• Please use the scoring rubric (Microsoft Word 55kB Apr20 13) as a guide for each category in the review, so that reviews are consistent across reviewers and across Cutting Edge. When you are done with your summary evaluation, be sure to click Submit in the Review Tool to submit your review.
  
• Conflicts of interest: please let us know if we have inadvertently assigned you one of your own activities, or if there are other conflicts of interest that were not apparent to us. Also, let us know if you are having any problems accessing the review tool or conducting the review itself.
  

What happens after you submit your review

• Each activity will receive two peer reviews before the team meeting. During the review team meeting, we will discuss conflicting reviews and decide on the final status of each activity.
  
• At the end of the review process across the Cutting Edge collections, we will have realized the long-term goal of NSF and the geoscience community of having a peer-reviewed collection of teaching activities!
  

Editorial Team

• Devin Castendyk, Managing Editor
• Maddy Schreiber, Associate Editor

Associate editor page