Reviewed Activities in On the Cutting Edge

Many of the teaching activities on the Cutting Edge website have undergone some kind of peer review. Much of this review has taken place as a part of a workshop (either face-to-face or virtual) whereby participants have given each other feedback on the activities they have submitted as a part of the workshop. But in 2011, On the Cutting Edge also piloted a peer review process leading up to a workshop on Teaching Mineraolgy, Petrology, and Geochemistry in the 21st Century. Find more information below on the various ways that the activity collections have benefited from input from educators beyond the author.

Jump down to: Cutting Edge Peer Review Process | Workshop Review Process | User Review Processes | Other Review Activities

Cutting Edge Peer Review Process

Cutting Edge began conducting a comprehensive review of all the teaching collections in 2011 with the goal of developing a comprehensive and coherent set of teaching activities that will serve the geoscience education community for years to come. The initial implementation of the review process was in conjunction with the Teaching Mineralogy, Petrology, and Geochemistry in the 21st Century workshop and all teaching activities related to these three topical areas were the first to undergo the process. As instituted, the process involves having workshop participants review a set of teaching activities, using a rubric (Acrobat (PDF) 27kB Oct19 12) to help calibrate their scoring. There are five elements of each activity that get reviewed during the process:

• scientific veracity
• alignment of goals, activity, and assessment
• pedagogical effectiveness
• robustness (usability and dependability of all lesson components)
• completeness of the ActivitySheet web page for the activity

All of the review is conducted online via a web-based management tool, with each participant being assigned 5-10 activities to review. As the participants do their reviews, they enter scores for each element and provide constructive comments to help the author address any noted deficiencies. Each activity receives two independent reviews. The review process is managed by the Cutting Edge PI Team and a number of associate editors selected from the workshop community. The associate editors maintain communications with the reviewers to ensure that the process moves ahead smoothly. The PI Team has the ultimate responsibility for assigning the final ratings and communicating the activity authors. Based on the results of the reviews, each activity receives one of four possible ratings:

• Exemplary: Activities in the Exemplary Collection have received Exemplary or Very Good scores in all five categories and must have been rated Exemplary in at least 3 of the 5. It is expected that no more than 10-20% of the activities in the collection would be awarded an Exemplary rating in this process.
  
• Reviewed: Activities in the Reviewed Collections have received positive reviews in all five categories, consisting of mostly of Very Good scores with possible Exemplary or Adequate scores in one or more areas. Authors with activities in this collection will receive recommendations from the reviewers and associate editors for ways of improving their activity such that it can be brought into the Exemplary Collection.
  
• Activity Idea: An activity with this rating contains the nucleus of a good teaching activity in the materials that were submitted, but in its current form does not contain sufficient information to be able to be widely used in geoscience classes. Authors will be encouraged to invest energy in further developing the activity so that it become part of the Reviewed Collection.
  
• Deaccession: These activities contain serious deficiencies which would be difficult or impossible to remedy. They will be removed from the teaching activities collection entirely.

Workshop Review Process

The process by which activities are reviewed at On the Cutting Edge activity development workshops is similar regardless of whether it occurs at face-to-face or virtual workshops.

• Either before or during the workshop, participants submit an activity they have developed or adapted for use in their classroom.
  
• Participants review each others' activities and comment on them using a rubric. Information on the review criteria and rubrics used is available for participants on pages such as this one from the 2010 Teaching Services Learning in the Geosciences workshop.
  
• The reviewer and author discuss the reviewer's comments on the activity.
• Authors are encouraged to work on revisions to their activity based on the feedback they received both at and after the workshop.
  

Beyond necessary technical differences in the process, there are also functional differences between review at face-to-face and virtual workshops. Virtual workshops are typically made up of multiple synchronous sessions spread over the course of some extended period of time. The first several sessions are usually devoted to exploring the workshop topic in depth to give participants a grounding in the state of knowledge in the field. In between these synchronous sessions, participants are usually tasked with developing, reviewing, and/or revising teaching activities and other workshop products for the website. The last synchronous session(s) allow participants to showcase their work to the rest of the group and explore next steps.

User Review Processes

Users of the On the Cutting Edge website are also invited to review activities on the website. If you would like to review activities using the rubric as done in workshops, see the online version of the rubric form. Another option for reviewing activities is to run them in your classroom and observe how well they work. The Using MARGINS Data in the Classroom project has developed an Observational Protocol which has been adapted by On the Cutting Edge for use in this kind of review. To learn more about the protocol, see the Assessing Activities in Action page.

Other Review Activities

A body of teaching activities within the Mineralogy topical area were reviewed for publication in

Brady, J., Mogk, D. W., and Perkins, D., (editors), 1997, "Teaching Mineralogy," a workbook published by the Mineralogical Society of America, 406 pp.

All teaching activities in this volume received two external peer reviews from mineralogy faculty focused on content and pedagogy, and a final review by the co-editors to comply with the publication standards of the Mineralogical Society of America.