Initial Publication Date: August 31, 2020

Findings from Student Work

To evaluate student learning in the field course, the SERC evaluation team analyzed data from pre and post student surveys and student interviews and student work products. The work products consisted of scores from the following six assignments designed to tap students' mapping and field notes skills.

These activities were scored with the mapping rubric and the field notes badge rubric, respectively. Both rubrics are described in more detail below. The rubrics underwent substantial revisions from 2017 to 2018. As a result, only student work from 2018 was included in the analyses.

Two key findings emerged from analyses of student work:

  • Students' scores on the mapping activities indicated improvement over time, with fewer students not meeting the standard as the course progressed.
  • Students' scores on the field notes activities were more mixed, but indicated improvement over time from the first to the second assignment.

Mapping Assignments

Students completed SCZO Soil Maps, Walker Lake Area Maps, and Little Poleta Geologic Maps on Day 4, Day 10, and Day 19 of the field camp, respectively. Students' maps were scored in four different categories:

  • Map Elements: the student includes general map features (e.g., name and north arrow) and discipline-specific features (e.g., folds and faults).
  • Thoroughness and Legibility: the map is complete and legible.
  • Accuracy: the map and map-related interpretations reflect accurate field observations.
  • Justification of Interpretations: interpretations are supported by field observations, areas of uncertainty are specified, and the map is used to draw conclusions.

More information about these categories can be found in the rubric for the Mapping Badge. The mapping rubric was designed to allow instructors to score student work at three levels: did not meet standard, met standard, and exceeded standard. In practice, however, a "nearly met standard" grading category emerged during the grading process. To more accurately reflect changes in student performance, this category was included in the analyses.

Figure 1 presents the percentage of students who received each score for the four mapping categories. Because the Little Poleta Geologic Map did not require interpretation, students did not receive a score for the "justification of interpretations" category for this assignment. (click the image to enlarge) As shown in Figure 1, students' scores on these mapping activities indicated improvement over time. Specifically, the proportion of students who did not meet the standard decreased -- oftentimes substantially -- in each mapping category as students progressed through the field course.

Field Notes Assignments

Students completed the SCZO, Panum Crater, and Bristlecone field notes on Day 4, Day 12, and Day 21 of the field camp, respectively. Students' field notes were assessed in the following eight categories on a three-point scale: did not meet standard, met standard, and exceeded standard.

  • Interpretation: the interpretations are supported by data and are scientifically plausible.
  • Useful for Interpretation: the student uses their recorded observations to make correct interpretations.
  • Data: the data is recorded correctly, accurately, and completely.
  • Figures: figures are neat and legible, and include north arrow, scale, labels, and keys when appropriate; figures clearly differentiate data and interpretation.
  • Thoroughness: important site-specific information, errors, and assumptions are clearly recorded.
  • Inclusion of Components: the student includes elements such as name and field partner.
  • Recollection: Others: others are able to understand the field notes.
  • Organization and Clarity: the table of contents is updated and all the information is easy to find.

More information about each of these categories and the remaining categories not used in these assignments can be found in the rubric for the Field Notes Badge. A "nearly met standard" grading category emerged during the grading process and again is included to accurately reflect student performance. The figures below show the percentage of students at the different points on the grading scale for each of the field notes categories. The Activity 1 (SCZO) field notes did not require interpretations and were not scored in the "interpretation" or "useful for interpretation" categories.

As shown in Figure 2, the percentage of students who did not meet the standard decreased in the "interpretation," "useful for interpretation," and "figures" categories as students progressed through the field course. (click the image to enlarge) In the "data" category, the percentage of students who did not meet the standard decreased from Activity 1 to Activity 3, but stayed at the same level at Activity 5.

Figure 3 shows the "thoroughness", "inclusion of components", and "recollection: others" categories. In each of these categories, students were again assessed on a scale of standard not met, standard partially met, standard met, and standard exceeded. (click the image to enlarge)

Finally, Figure 4 shows the "organization and clarity" category. This category was assessed on a standard not met, partially met, or met scale. (click the image to enlarge)

As shown in the Figures 3 and 4, the percentage of students not meeting the standard decreased -- oftentimes substantially -- from Activity 1 to Activity 3 and then modestly increased from Activity 3 to Activity 5 in the remaining four categories. The modest increase in the percentage of students not meeting the standard may be the result of a number of factors. One possibility is that the scores reflect a real decline in student performance in these four categories. An alternative explanation is that the scores reflect changes over time in how student work was scored or in the nature of the activities. 

Specifically, these changes (and others presented here) should be interpreted with some caution given that

  • there were often different scorers for different activities;
  • some activities (e.g., Activity 5) were longer than the other activities which may have led students to spend less time than would be ideal on some elements of the assignment; and
  • some activities (e.g., Activity 5) had a less favorable student-to-instructor ratio than other activities which may have interfered with students' ability to seek out assistance if they encountered difficulties.