Initial Publication Date: July 28, 2017

EDDIE Environmental Data: Assessment

Learning Goals

There are four overarching learning goals that we hope to address with EDDIE modules.

  • Inquiry & Problem-solving – We seek to develop students' ability to conduct inquiry and solve problems using big data.
  • Quantitative Reasoning – We seek to enhance students' quantitative reasoning about statistical concepts, such as sampling and variation.
  • Nature of Science Understanding – We seek to develop students' understanding of the nature of science, particularly natural sciences that do not employ tightly controlled experiments in lab settings.
  • Scientific discourse – We seek to foster students' ability to engage in scientific discussions in which they verbalize evidence-based reasoning.

Assessment Process

In order to assess the efficacy of EDDIE modules in achieving these goals, we are employing an iterative design process. During the 2014-2015 academic year, we administered pre/post-tests comprised of questions that measured the first three constructs of interest. To measure inquiry ability, we modified open-ended items on the Experimental Design Ability Test1 so that questions were more proximal to content taught in modules. To measure quantitative reasoning, we used the Distributional Variation Subscale (DVS) and additional items from a previously developed instrument that examines statistical ideas2 and items that asked participants about their comfort using Excel. To measure nature of science understanding, we administered items from the Views on the Nature of Science, form C3. We recorded three student groups as they collaboratively completed a module to examine the discourse that was inspired by the module.

Open-ended responses of Year 1 pre/post data were coded by a team of three researchers. Categories of responses were used to create response options for close-ended versions of the same questions. These resulting close-ended items, in which the answer options reflect common responses provided by students, are being administered in the same pre/posttest fashion during the 2015-2016 academic year.

Recordings of students collaboratively completing modules revealed that students do not engage in much discourse while completing modules unless explicitly prompted to do so. Because much module completion is performed on a computer, if it is done in pairs, only one student can do the work on the computer at a time, relegating the other to the role of onlooker.

Preliminary Findings

Based on Year 1's data, we suggest the following tips to instructors to maximize teaching efficacy when using EDDIE modules:

  • While students and instructors alike recognized a genuine struggle with becoming familiar with using and Excel, Excel comfort and ability significantly increased after just one module. To minimize this frustration, instructors may want to forewarn students of the struggle while at the same time acknowledging the necessity of learning spreadsheet skills in order to conduct basic inquiry.
  • Because such little discussion among students was observed in video-recordings, instructors may want to make a point to facilitate discussion among students when completing modules collaboratively. Alternatively, instructors may want to assign modules to individual students and then generate discussion of results obtained by students who completed the activity independently.
  • After completing an EDDIE module, students were more likely to use big data to solve a scientific problem when asked. Instructors may want to have explicit discussions with students about the utility of big data and its use in answering a variety of scientific questions.

References

1 Sirum, K., & J. Humberg. (2011). The experimental design ability test (EDAT). Journal of College Biology Teaching 37:8-16.

2 Watson, J.M., B.A. Kelly, R.A. Callingham, & J.M. Schaughnessy. (2003). The measurement of school students' understanding of statistical variation. International Journal of Mathematical Education in Science and Technology 34:1-29.

3 Lederman, N.G., F. Abd-El-Khalick, R.L. Bell, & R. Schwartz. (2002). Views of nature of science questionnaire: Toward valid and meaningful assessment of learner's conceptions of nature of science. Journal of Research in Science Teaching 39:497-521.


« Previous Page