"Intellectual Merit" and "Broader Impacts" criteria are of equal significance in the consideration of proposals submitted to the National Science Foundation. The National Alliance for Broader Impacts developed a guiding document as part of its efforts to facilitate the design of initiatives that meet National Science Foundation's "Broader Impacts" merit review criteria (NABI, 2015). This study will quantify and compare broader impact trends within the four NSF Geosciences Divisions (Atmospheric and Geospace Sciences, Earth Sciences, Ocean Sciences, and Polar Programs) by leveraging 2012-2018 outcome reports. Theory- and research-informed strategies to support broad and impactful STEM education efforts will be proposed in response to emerging trends.

Many of humanity's most pressing issues are socio-environmental, and as such, are ill-structured, "wicked" problems. Investigating wicked problems requires a collaborative process that includes researchers from different disciplines and stakeholders with different perspectives. A key challenge in working on these problems is that the problem scope is typically unbounded, the issues are complex and interwoven, and the problem-solving approach can be framed in a multitude of ways. This presentation will outline a new method called EMBeRS (Employing Model-Based Reasoning in Socio-Environmental Synthesis) for facilitating collective learning during the early, formative phases of collaborative problem solving. The EMBeRS method is based on a synthesis of learning, cognitive, and social theories, including: 1) constructivism, 2) experiential learning, 3) model-based reasoning, 4) boundary negotiating, 5) epistemic objects, and 6) distributed cognitive systems. This presentation will describe the EMBeRS method and its theoretical basis, its pilot application in a ten day workshop for PhD students nationwide held in 2016 and 2017, and the results from evaluation of the workshop. The workshop facilitated students working in teams to formulate interdisciplinary research conceptualizations on water and agricultural systems. Evaluation data indicates that the workshop dramatically improved participants' transdisciplinary orientation and developed skills and competencies unavailable through their regular programs.

TRESTLE (TRansforming Education, Stimulating Teaching and Learning Excellence) is a 5-year, multi-institution, NSF funded project that began in Fall 2015 and is working to implement and evaluate a model to promote improved STEM education at research universities. There are a total of seven universities participating in the project, including the University of Kansas (lead), University of Colorado Boulder, Indiana University, University of California Davis, University of Texas at San Antonio, Queen's University, and the University of British Columbia. Each institution is testing a local adaptation of a model with three core components: (1) support discipline-based educational experts in departments to catalyze course transformation, (2) build intellectual communities around evidence-based educational improvement, within and across departments and institutions, and (3) collect and make visible evidence of the impact on teaching and learning. Each campus is collecting a variety of assessment measures in order to evaluate the effectiveness of this model at the local institutions. This includes qualitative information from the experts and faculty with respect to the changes being implemented within individual courses, analysis of the syllabi to evaluate the distribution of time during the semester among different categories of assignments and learning activities, and lastly multiple campus wide surveys about teaching practices, attitudes and climate. In addition, classroom observations using the COPUS tool are being used to provide quantitative measurements of changes happening in the classroom, as well as DFW rates to gauge student learning and success. This presentation will highlight some of the major successes of this project to date, including a review of the number of courses and faculty impacted by the work, an evaluation of the COPUS data for courses being transformed and comparison courses, and lastly a case study review of the impact on DFW rates for courses being transformed at the University of Kansas.

We investigated the impact of geoscience-specific professional development (PD) on undergraduate teaching by comparing instructor PD histories and classroom practices. Trained observers used the Reformed Teaching Observation Protocol (RTOP) in 236 geoscience classes at both the introductory and upper-level. Total RTOP scores range from 13 to 89 (avg = 39.0), and were used to categorize classroom instruction as: Teacher Centered (≤30; 31% of observations), Transitional (31–49; 46%), or Student Centered (≥50; 22%). For each observed instructor, we compiled a pre-observation history of participation in On the Cutting Edge, InTeGrate, and National Association of Geoscience Teachers PD. Instructors who attended at least one PD event (n=111) have higher average RTOP scores (44.5 vs. 34.2) and are more frequently observed teaching Student Centered classes (33% vs. 13%) than instructors with no PD (p < 0.001). To examine the influence of workshop content on teaching practices, we compared RTOP scores with instructor participation in workshops that could be aligned topically with observed courses. Each instructor with PD was classified as: having topically-aligned PD if the instructor attended a topical workshop and was observed teaching that same topic; having potentially topically-aligned PD if they attended a topical workshop but were observed teaching a different topic, or; having non-topically-aligned PD if they attended workshops on topics not associated with disciplinary content. The number and alignment of PD events have an impact on RTOP score. Each PD event attended improves the RTOP score by 1.34 points; attending a topically-aligned PD event improves the RTOP score by 13.5 points (p<0.0001). Attending topical workshops is strongly correlated with teaching Student Centered classes. One topically-aligned workshop or two or more potentially topically-aligned workshops make you approximately 5.5 times more likely to teach a Student Centered class than those without that PD.

Nationwide, there is a growing emphasis on effective undergraduate geoscience education. A central element of geoscience teaching and learning involves scientific modeling and systems thinking (SMST). While studies of individual courses or instructional interventions may provide empirical insights into SMST in geoscience education, few efforts have attempted to document where, when, why, and how undergraduate geoscience courses emphasize SMST elements, as well as factors that can help explain and/or predict these trends. Data for this study is from the National Survey of Geoscience Teaching Practices produced for the National Association of Geoscience Teachers by the NSF-funded professional development program, Cutting Edge, for post-secondary geoscience faculty. First administered in 2004, administration of the survey occurred four times over the past 14 years. The present study reviews results from the most recent (2016) administration of the survey (n=2056). We investigated instructor- and course-related variables as they related to a set of nine survey items that serve as the measure for SMST. Reported individual SMST practices varied significantly, with frequencies varying from <20% to >75%. Total reported SMST course elements varied significantly by faculty sub-discipline, use of innovative pedagogical practices (i.e., active learning), recent changes to both the content and teaching of courses, and overall engagement in instructional improvement. Both geoscience research and education-oriented faculty reported similar emphases on SMST course elements that were significantly higher than non-tenure line, teaching faculty respondents. A linear regression model contained predictor variables identified in the analyses. This model was able to account for 17% of the variance in reported SMST course elements, F(9, 2010) = 21.12, p < .001, R2 = .17, 95% CI [.69, 2.3]. These findings have important implications for the design of undergraduate geoscience courses to foster student learning through SMST.

Quantitative tools and data analysis techniques are important for the success of geoscience majors in a variety of careers, and are important for many non-scientists who need quantitative reasoning skills for succeeding in society. In four administrations of a national survey of geoscience faculty in 2004, 2009, 2012, and 2016, faculty reported on the extent to which they included quantitative and data analysis skills in a specific course at either the introductory or majors level. Over 2000 faculty responded to each survey administration. The 2016 survey results show that for majors courses, 87% of faculty (n = 1066) asked students to use algebra, 58% asked students to use statistics, and 45% asked students to use calculus. The reported use of statistics decreased between 2009 and 2012-2016, but there was a slight increase in the reported use of algebra and no change in the reported use of calculus. In 2016, 61% of faculty asked students to collect and analyze their own data, 74% asked students to address uncertainty when interpreting data, and 66% asked students to evaluate assumptions in estimation, modeling, or data analysis. Faculty asked students to use quantitative and data analysis skills less frequently in introductory courses. Only 35% of faculty (n = 1096) asked students to use statistics, 75% of faculty asked students to use algebra, 41% of faculty asked students to collect their own data, and 44% asked students to evaluate assumptions. These results indicate that the majority of geoscience majors courses include quantitative and data analysis skills, but there is room to enhance the focus on technical skills to better prepare students for the workforce. There is also a need to increase the use of these skills in introductory courses to improve students' success in the geoscience major and to strengthen quantitative reasoning for non-scientists.

Most teachers experienced Earth science in a single introductory undergraduate geoscience course. To what extent do classroom practices in introductory geoscience courses reflect the practices we expect future teachers to use in their classrooms, as defined in the Framework for K-12 Science Education and standards based on the Framework? I make use of results from the National Geoscience Teaching Practices survey, which has been administered four times: in 2004, 2009, 2012, and 2016. The survey includes questions about teaching methods, and the extent to which they incorporate quantitative and data analysis techniques, geospatial skills, and systems thinking skills, among others. I analyze responses for introductory courses (n = 775 in 2004, 966 in 2009, 909 in 2012, 1096 in 2016). Teaching methods: Results show that respondents who spend 20% or more of class time in the "lecture portion" of their introductory course on student activities, questions, and discussion was lowest in 2004 at 41.3% and highest in 2016 at 67.0%. The percentage of respondents who indicated that students collected and analyzed their own data was highest in 2016 at 41%. I use caution in interpreting changes over the four administrations as increases, because the audience reached by the survey changed as well. Skills: While the use of algebraic equations in introductory courses is quite common, statistical analyses are rarely used. Similarly, working with geospatial data is common, while making a geologic map is not. A small minority of faculty incorporate aspects of systems thinking such as making systems visible through causal maps and making predictive models. These analyses suggest that the coupling between the college-level geoscience courses that future teachers are most likely to take and what they are expected to teach is loose, and should be considered in development of both introductory geoscience courses and professional development for teachers.

The authors present the design, research, and evaluation of an earth science course for preservice elementary teachers that is place-based, hands-on, technologically enriched, and aligns with the Next Generation Science Standards (NGSS). Our redesigned earth science course was developed using the three-dimensional framework of the NGSS, emphasizing the practices of science and the use of Internet-based environmental data. The overarching theme of earth system processes is exemplified within a single stream watershed; each unit is tied to observations the students make in the field and online. In the first two years of the project, we have collected baseline data from the traditional version of the course, a large lecture class serving students in multiple majors; piloted the new curriculum in a small course with education majors; and then scaled up elements of the revised curriculum to a large lecture course. The delivery of the redesigned course is intended to impact preservice elementary teachers by increasing their learning and skill development, bolstering their confidence in their ability to teach science, and increasing their motivation to learn science. We discuss the ongoing research efforts to document impacts using a combination of validated measures, observation protocols, and document analysis. We present evidence of positive changes in instructor and student activities aligned with active learning and the NGSS practices, as well as challenges faced in measuring the effects of curriculum redesign efforts on student knowledge, beliefs and motivation.