Are we testing what we think is important? An historical, exploratory analysis of the New York Regents Exam in Earth science
Oral Session Part of
Thursday B: Program and Faculty Development
Authors
Anne Egger, Central Washington University
Michael Alvernaz, Central Washington University
Ryan Carrington, Central Washington University
Anthony Perez, Central Washington University
Over the last decade, communities of scientists came together to develop the literacy documents in climate, oceans, atmospheric science, and Earth science. These documents outline what every citizen should know to be "literate" in these areas. The literacy principles encapsulated in each of these documents were taken into account in the development of the Next Generation Science Standards, where, presumably, they will guide the development of what is taught and tested in adopting states across the country.
We asked the question: to what extent are those literacy principles reflected in current high school assessments? At the high school level, are we testing the concepts that the scientific communities think are most critical for all citizens to know? To address this question, we examined the longest-running Earth science assessment in the country: the New York Regents Exam.
We examined one exam from each decade starting in 1942 (the second year of the Earth science exam) and up to 2012. We coded exam questions for alignment with the literacy principles, "sphere" addressed (lithosphere, atmosphere, solar system, etc.), level in Bloom's taxonomy, question type (multiple choice, true/false, etc.), quantitative skills used, and whether or not the reference tables are required. Several initial observations emerge from this analysis: very few of the literacy principles are addressed in the exam in general; significant changes in all aspects of the exams occurred between 1962 and 1972, and between 1993 and 2002. While the changes in the 1960's can be attributed in part to the establishment of plate tectonic theory and the space race, the changes in the 1990's are not as easily explained. If New York State adopts the NGSS, significant changes will be required to align the Earth science Regents courses and exams with the standards.
We asked the question: to what extent are those literacy principles reflected in current high school assessments? At the high school level, are we testing the concepts that the scientific communities think are most critical for all citizens to know? To address this question, we examined the longest-running Earth science assessment in the country: the New York Regents Exam.
We examined one exam from each decade starting in 1942 (the second year of the Earth science exam) and up to 2012. We coded exam questions for alignment with the literacy principles, "sphere" addressed (lithosphere, atmosphere, solar system, etc.), level in Bloom's taxonomy, question type (multiple choice, true/false, etc.), quantitative skills used, and whether or not the reference tables are required. Several initial observations emerge from this analysis: very few of the literacy principles are addressed in the exam in general; significant changes in all aspects of the exams occurred between 1962 and 1972, and between 1993 and 2002. While the changes in the 1960's can be attributed in part to the establishment of plate tectonic theory and the space race, the changes in the 1990's are not as easily explained. If New York State adopts the NGSS, significant changes will be required to align the Earth science Regents courses and exams with the standards.