Assessing validity of the Geoscience Literacy Exam (GLE)

The Geoscience Literacy Exam (GLE) consists of 52 multiple choice and 30 essay questions that align with the content presented in four literacy documents: Earth, Ocean, Atmosphere and Climate. For each literacy principle, the GLE has a set of tiered questions that target increasing cognitive levels. Level 1 multiple choice questions are considered within the "remembering and understanding" level, while Level 2 multiple choice questions are multi-select and within the "applying and analyzing" cognitive level. A third level of questions are essays that focus on analysis. Item analysis of student responses (n=1275+) and qualitative feedback from expert review (n=12) were used to determine the validity of items composing the GLE. Revisions incorporated best practices for design of multiple choice items and altered test language and format based on the data collected. Revised questions were grouped according to the content of the literacy documents and the cross-cutting themes (i.e. "Bigger Ideas") developed by Duggan-Haas and Clark (2009) to meet the needs of the interdisciplinary nature of the curricula designed through the NSF-funded InTeGrate Project. The next phase of the assessment design will continue to focus on item validity as well as on the reliability of the revised multiple choice questions as the InTeGrate curricula and GLE are tested more widely among institutions across the nation. The presentation will provide an overview of the question design, item analysis, and further information about the research project. Members of the assessment team also include Stuart Birnbaum, Leilani Arthurs, Aida Awad, Barbara Bekken, Susan Buhr, Josh Caulkins, Kristin O'Connell, Megan Plenge, Mary Savina, and Karen Viskupic.