Site Review

Initial Publication Date: November 14, 2019

NAGT's portal, Teach the Earth, is a collection of all things relevant to geoscience educators, including teaching activities, research on learning, pedagogical strategies, and more. Within Teach the Earth, teaching activities are reviewed through the Activity review process and project sites are reviewed through the process described here.

Why review sites?

There are four goals we can achieve through peer review of project sites:

  1. Site reviews provide guidance for users who visit Teach the Earth. Sites (and activities) that pass review and/or are reviewed as exemplary show up higher in search returns, and are visibly tagged with their status.
  2. Reviewers' comments can be shared back to site authors and developers, who may choose to revise their materials to improve their sites.
  3. The process of reviewing sites helps reviewers develop better websites in the future, much in the way that being on an NSF review panel or editorial board functions.
  4. Finally, the rubric and guidelines for meeting the rubric can be shared with future site developers so that they have a target for creating an exemplary site.

How does it work?

Each site is assigned to three independent peer reviewers with knowledge and expertise in the content area and/or target audience of the site. The reviewers score the site according to a rubric, shown below, and provide comments that can be shared with the creators of the site. Based on the scores for all of the criteria, a final score is calculated by the review system, where exemplary = 4 pts, very good = 3 pts, adequate = 2 pts, and problematic = 1 pt. The total possible score is 16. The final rating is based on all three reviewers scores and the editor's decision, but generally a score of 15 or 16 is required for the site to be exemplary and scores must be above 12 to pass. Sites with scores lower than 12 are kept within Teach the Earth, but are not featured or highlighted.

An editor compiles the reviews, determines whether the site passes the review, is exemplary or not, and communicates the results of the review with the site authors. If the site authors choose to make changes to the site based on the review, the editor will review those changes and can change the status of the site (from "pass" to "exemplary," for example).

The rubric

The criteria and sub-criteria are shown below. If you are a site developer, please take a look at our guidance for Meeting the rubric.

Criterion 1: Relevance to the NAGT Audience

This criterion has three sub-criteria that will be scored and will sum up to the total:

  • Is the audience for the project well-defined?
  • Are the topics and strategies described relevant to the NAGT membership or some subset of the membership?
  • If local or regional projects, are they relevant/adaptable to other regions?

Criterion 2: Grounding in Research and Best Practices

This criterion has four sub-criteria that will be scored and will sum up to the total:

  • Is the context of and need for the project made clear?
  • Do the overall goals of the project reflect what we know from the research on learning and best practices in teaching?
  • Does the project make effective use of those best practices?
  • Does the project include appropriate and credentialed references?

Criterion 3: Robustness

This criterion has four sub-criteria that will be scored and will sum up to the total:

  • Is the site well-organized, easy to navigate, and is it clear what you are going to get when you go places?
  • Are the resources provided sufficient to inform and support the new user?
  • Does the quantity, depth, and breadth of the available resources match the scope of the project?
  • Is there alignment between the context, goals, and resources?

Criterion 4: Accuracy and currency

This criterion has three sub-criteria that will be scored and will sum up to the total:

  • Are the project materials scientifically accurate and free from misleading statements?
  • Is the project site actively maintained?
  • If not currently maintained, are the materials of an enduring quality?