Improving Programs

Part of the InTeGrate California State University - Chico Program Model

Program Motivation

Integrating societally relevant curriculum in the Sustainability GE Pathway at CSU, Chico

The General Education (GE) Program at CSU, Chico was redesigned in 2012 as a series of Pathways that link GE courses within a common theme. Pathways include lower and upper division courses that allow students to consider broad topics throughout their undergraduate education and provide common intellectual experiences (e.g. Kuh, 2008). Students select from courses options in their chosen pathway to fulfill lower division requirements in areas that are a) foundational (e.g. writing, math, science) and b) disciplinary (Arts, Humanities, Societal Institutions, Lifelong Learning) and c) one course each in upper division areas of Natural Sciences, Arts/Humanities and Social Sciences. The faculty who teach courses in the Sustainability Pathway recognize the Implementation project as a unique opportunity to work together to make the pathway more cohesive and to improve student learning with the coordinated use of the InTeGrate curriculum modules.

Introduction

Our initial vision for the end state of this project was to develop a program that could serve as a model for the General Education Pathways. The model would be founded on improving student learning and student engagement as well as faculty collaboration and peer professional development. Results of our work are consistent with this goal, in that we have developed this model and are optimistic that similar programs can be developed based on successes (and learning lessons) from the Chico InTeGrate Implementation Program.

Program-Level Goals and Evidence

Goal 1: Measurably improve student learning and confidence with InTeGrate curriculum in the Sustainability Pathway

A pre- and post- curriculum survey was administered to students in the Chico IP to measure their gains in content knowledge as well as in their self-confidence with the curriculum. The survey was developed by participating faculty such that students who participated in the same content were given the same questions, but only questions relevant to material covered.

Data were collected prior to and following curriculum in each course and compiled for comparison of learning gains related to the curriculum. Learning gains can be compared within each course and among other courses to evaluate changes related to course level (e.g. upper/lower division), course topics (e.g. science/non-science) and across semesters in which data were collected.

Content Data:

Content knowledge was measured with our pre- and post- curriculum surveys, with questions specific to material covered. With multiple units from four modules included in this project, our evidence of learning is presented here with results from several units from the Climate of Change module.

Pre-curriculum surveys reveal:

1. Students in lower and upper division science classes have pre-existing knowledge of topics such as climate models, types of data used to model climate change, and most (>70%) are well versed in reading graphs of data.

2. Students in upper division non-science courses have lower scores in the use of data (than science students) but have higher pre-curriculum knowledge of societal impacts or content (e.g. adaptation and mitigation)

Learning gains from pre- to post- curriculum survey comparisons reveal:

1. Large learning gains occur for questions asking for details of climate models, the role of greenhouse gases on climate change, and for upper division students on the use of specific data and graphs that depict large data sets.

2. Small or negative learning gains occur in lower division science classes when asked about societal impacts and the use of graphs that depict large data sets.

3. Students in lower and upper division courses taught by the same instructor in the same semester all had small learning gains on a question asking them to, "describe the system processes (i.e. physical characteristics) that cause the cycling of the ENSO." Only 11% of lower division students and 18% of upper division students were able to answer the question correctly in the pre-curriculum survey, and learning gains were only 11% and 33% for the lower and upper division students, respectively- even when other questions resulted in medium and high learning gains for both populations of students. These results reinforce that complex processes merit extra attention during the Climate of Change unit, and likely need additional coverage in other course activities in order for students to achieve high learning gains.

Results from these and data from other InTeGrate modules suggest:

1. Our students are learning complex content (models, large data sets) but in many cases, upper division students are learning this more thoroughly (e.g. larger Learning Gains)

2. Students in science courses have greater prior knowledge and learning gains than non-science students have about types of climate data, reading graphs, and using models

3. Students in non-science classes have stronger prior knowledge than science students of societal impacts of climate change

Confidence Data:

Students were asked to use a Likert Scale (1-5) to rate their confidence in completing an activity similar to what they learned in InTeGrate modules. For example, one of the items students who participated in Climate of Change were asked is: I am confident in using climate information to identify how climate change could impact human societies. Our results indicate that pre-curriculum ratings in all classes are lower than post-curriculum ratings, indicating students gained confidence from InTeGrate materials.

More subtle variations among courses include that:

1. Students in non-science lower division courses have much lower pre- and post-curriculum ratings than students in a lower division science course.

2. Among non-science courses, students in lower division courses have lower pre- and post-ratings than non-science upper division courses and

3. Post-curriculum ratings in non-science upper division courses are comparable to post-curriculum ratings in lower division science courses.

Items 1-3 suggest that students in non-science courses gain the confidence of science students only with additional academic maturity (e.g. upper division status), but that this higher level of confidence is achievable by non-science students.

Faculty observations of student learning are subjective, but illustrate observations of students in the classroom not captured by student learning surveys:

• The modules are designed for a strong emphasis on student engagement and participation...
• Students seem to retain concepts much better with this approach

Goal 2: Develop a community of faculty who collaboratively incorporate InTeGrate resources in pathway courses and work together in peer-based professional development.

Faculty participation in the Chico Implementation Project included multi-day workshops (in summers) and nearly weekly activities meeting together, observing one another's classes, being observed, or debriefing peer observations. Research meetings also include review and analysis of student learning data. Faculty were also asked to complete self-reviews after teaching InTeGrate curriculum and an end-of-semester survey with questions regarding their participation in the project.

Data from faculty surveys indicate that this project has succeeded in developing a collaborative faculty learning community with the following data:

• Item 6: Rank the impact has the IS project had on your communication about teaching (content and pedagogies) with your colleagues, a) Within the IS team; b) Within the Pathway faculty and c) among all other colleagues.

Responses are rankings from 1-5 (from very positive to very negative impacts) and all faculty report either "very positive" or "some positive" impacts. Results from the first two semesters of the project (fall 2015 and spring 2016) indicate that the FLC has had a positive impact on faculty communications about teaching with each other, and the communication has increased from fall 2015 to spring 2016. The FLC has also had some positive impact on communications about teaching with other colleagues, but is stronger among the FLC cohort.

In terms of personal growth through the Implementation Team's 2^nd goal to build a strong Faculty Learning Community, the faculty survey asks:

• Item 4: What impact has the IS content had on your teaching practice in general?
  
• Item 5: What impacts has the IS pedagogies had on your teaching practice in general?
  

As with item 6, responses are rankings from 1-5 (from very positive to very negative impacts) and all faculty report either "very positive" or "some positive" impacts and responses become more positive (e.g. from "some positive impact" to "very positive impact" from fall 2015 to spring 2016.

From responses 4, 5 and 6, the data shows (and we collectively agree!) that the faculty learning community has been an important and successful component to this project. Open ended comments from faculty surveys also illustrate the collaborative nature of the project:

• Previously I had no contact with other instructors in my pathway. I now have some understanding of the content they are teaching and the ways they try to introduce material to their students.
• Our discussion in the IS team meetings and visiting classes has been very useful for getting a better understanding of other courses
• (The project) ... has provided new and interactive ways to deliver the information... it remains interesting to see how the different courses are using the material.

Unexpected Outcomes

Student learning data from content surveys include reasonable outcomes (e.g. lower division students have smaller learning gains for more complex themes than upper division students), but also revealed to us a higher level of prior knowledge than we expected. For instance, as reported above,

• High scores on pre-curriculum surveys revealed that students in lower and upper division science classes are more knowledgeable of topics such as climate models, types of data used to model climate change, and most (>70%) are well versed in reading graphs of data.

Another unexpected outcome is the disparity between prior knowledge (or pre-curriculum scores) of students in science courses vs. those in non-science courses. As reported above,

• Students in upper division non-science courses have higher pre-curriculum knowledge than students in science courses on the societal impacts or content (e.g. adaptation and mitigation)

Long-term Impact and Next Steps

Making broad changes to courses that result in more coordinated curriculum in GE Pathways requires multiple faculty from different departments and colleges within the university. Establishing the Implementation team in the Sustainability Pathway was facilitated by funding from the InTeGrate program, which provided time and motivation to develop and complete our course goals as well as to collect, compile and interpret our data. As a FLC, we continue to learn from each other and through our community of practice on ways to improve the pathway. Our current plans are:

• Within our team: 1) continue to use the InTeGrate curriculum in our pathway courses, and several in the team intend to adapt additional modules to pathway and other courses; 2) compile and interpret our data and disseminate those results in publications
• Beyond our current team: meet with additional pathway faculty to share results of the adaptation of curriculum and pedagogical strategies in our courses and to invite them to use InTeGrate curriculum in their courses and participate in our FLC.
• Beyond the Sustainability Pathway: continue to invite university administrators and pathway coordinators to attend classes using InTeGrate materials and our FLC meetings to learn more about the Chico Implementation Team as a model for other faculty groups to use in coordinating curriculum in other pathways

Thus, the experiences and insights gained in a project like this, can have wide reaching effects, beyond the scope of the project itself.