Activity Review Process

As part of the Teaching with Online Field Experiences project, authors submitted activities from April through July of 2020. The peer review process closely followed the model of the Teach the Earth review process, with a few questions added to the rubric (see below). A team of peer reviewers who were not involved in the authoring reviewed each activity for five elements:

• Scientific accuracy
  • Is the teaching activity free from scientific inaccuracies and misleading statements?
  • Does the activity cover scientific knowledge, concepts, skills, or methods that are important to the topic of the collection?
  • Does the activity link to, or cite credentialed scientific sources when appropriate (e.g. journal articles, facilities, ...)?
• Alignment of goals, activity, and assessment
  • Are the learning objectives or outcomes clearly articulated/defined? Note: learning goals may be focused on content or concept mastery, skill development, and/or affective aspects (dispositions, metacognition)
  • Are the instructional activities well-aligned with the learning goals?
  • Are suggested assessments of the learning outcomes aligned with the learning goals?
  • If a student completes the assignment, can the instructor determine whether the student has met the stated goals?
• Pedagogical effectiveness
  • Does the activity fulfill the components of experiential learning typical of a field-based experience?*
  • Is the resource engaging and motivating for students? Does it provide compelling reason to jump into the activity?
  • Does this activity/assignment help students place new knowledge, tasks, and experiences into the context of what they already know?
  • Does the activity enable students to engage higher order thinking skills (e.g. critical thinking, transference of knowledge, analytical or synthetic reasoning, problem-solving, inquiry, discovery...)? (as opposed to following cookbook steps, answering only leading questions, or doing "plug-and-chug" practice problems)?
  • Does this activity encourage students to reflect on their learning?
  • Does this activity/assignment require students to synthesize, discuss, extend, or reflect on what they have learned in the activity?
  • Is the activity accessible and inclusive for diverse audiences?*
• Robustness
  • Are the hardware (equipment) and software (programs/apps) required for this activity widely available or easy to procure?*
  • Do all required components of the teaching activity work reasonably well? (E.g., URLs required are up and running, datasets are reliable and available)
• Completeness of the activity description
  • Could this activity be easily used by someone else as it is currently presented?
  • Is there sufficient information for a new user to make an informed decision about whether or not to choose to use this activity in his/her class?
  • Is there adequate information to support the instructor in adapting the activity/assignment for his/her local context?
  • Are teaching tips provided that will help instructors to address identified barriers to learning (e.g. misconceptions)?
  • Are prerequisite skills, knowledge and understandings accurately indicated?

Questions with asterisks (*) were added to the basic review rubric for this collection. Reviews were conducted online via a web-based management tool and each activity received two independent reviews; a final rating was given by the managing editor based on the independent reviews. Based on the results of the reviews, each activity receives one of three ratings:

Exemplary: Activities in the Exemplary Collection have received Exemplary or Very Good scores in all five categories and must have been rated Exemplary in at least 3 of the 5.

Reviewed:Activities in the Reviewed Collection have received positive reviews in all five categories, consisting of mostly of Very Good scores with possible Exemplary or Adequate scores in one or more areas. Authors with activities in this collection will receive recommendations from the reviewers and associate editors for ways of improving their activity such that it can be brought into the Exemplary Collection.

Activity Idea:An activity with this rating contains the nucleus of a good teaching activity in the materials that were submitted, but in its current form does not contain sufficient information to be able to be widely used in geoscience classes. Authors will be encouraged to invest energy in further developing the activity so that it become part of the Reviewed Collection.

These rating are then reflected on the original web pages describing the activities. They are also displayed in search returns and highly rated activities are given more prominence in those search returns. The set of exemplary activities is also highlighted as its own distinct collection.

The work presented on this website was funded through a National Science Foundation RAPID grant (NSF-EAR 2029920) awarded to Kurt Burmeister, Laura Rademacher, and Katherine Ryker. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.