Evaluation is an important aspect of science and is receiving increasing attention in science education. This study investigated changes to high school students' plausibility judgments and knowledge as a result of a series of instructional activities that facilitated evaluation of scientific and alternative models in four different Earth science topics: (a) climate change, (b) fracking and earthquakes, (c) wetlands and land use, and (d) the formation of Earth's Moon. Repeated measure MANOVAs showed that participants shifted toward scientifically accepted explanations and increased their knowledge about these four Earth science topics after participating in the instructional activities. We also examined whether these evaluations mediated the relation between post instructional plausibility and knowledge. As shown by structural equation modeling, greater levels of evaluation significantly mediated plausibility shifts and knowledge increases. Effect sizes were small to large, depending upon topic and instructional context. Based on these results, the activities used in this study have the potential to facilitate students' critical thinking skills when evaluating the validity of explanations based on evidence, a scientific practice that is key to understanding Earth science.