Using Data to Improve Ecological Forecasts

×

Provenance: Tadhg Moore, Virginia Polytechnic Institute and State Univ
Reuse: This item is offered under a Creative Commons Attribution-NonCommercial-ShareAlike license http://creativecommons.org/licenses/by-nc-sa/3.0/ You may reuse this item for non-commercial purposes as long as you provide attribution and offer any derivative works under a similar license.

• Define data assimilation
• Generate an ecological forecast for chlorophyll-a in a lake
• Describe how to assess ecological forecast accuracy
• Describe how data assimilation affects forecast accuracy and uncertainty
• Explain how updating models with data collected at different time scales (e.g., daily, weekly) and with different levels of associated uncertainty affects ecological forecasts

This entire module can be completed in one 2 to 3-hour lab period, two 75-minute lecture periods, or three 1-hour lecture periods for introductory undergraduate students in Ecology, Environmental Science, Ecological Modelling, and Quantitative Ecology classes. This module can be coupled with other Macrosystems EDDIE ecological forecasting modules: Module 5 "Introduction to Ecological Forecasting"; Module 6 "Understanding Uncertainty in Ecological Forecasts"; or Module 8 "Using Ecological Forecasts to Guide Decision-Making". There are two versions of this module, depending on whether instructors wish to incorporate R coding into their course curriculum. For course curricula that do not include computer coding, instructors can teach the R Shiny version of the module, which is a point-and-click web interface with interactive data visualization. For course curricula that include computer coding and R, instructors can teach the RMarkdown version of the module, which will ask students to read and revise R code to complete module activities. We found that teaching this module in one longer lab section with short breaks was more conducive for introductory students than multiple 1-hour lecture periods. Please see the instructor manual for detailed recommendations about module timing for different class schedule types.

Quick overview of the activities in this module

See the instructor manual, provided below, for a step-by-step guide for carrying out this module. A student handout into which students can copy-paste plots and type their answers as they progress through the module and an introductory instructor PowerPoint are also provided.

• Activity A: Students access and explore data from a lake site of their choice in the National Ecological Observatory Network, then fit a model and generate a forecast of lake chlorophyll-a.
• Activity B: Students explore how updating model predictions with data affects forecast accuracy, including the effects of data observation uncertainty and temporal frequency.
• Activity C: Students explore the effect of assimilating sensor data at different frequencies on management decision-making.

Why macrosystems ecology and ecological forecasting?

Macrosystems ecology is the study of ecological dynamics at multiple interacting spatial and temporal scales (e.g., Heffernan et al. 2014). For example, global climate change can interact with localland-use activities to control how an ecosystem changes over decades. Macrosystems ecology recently emerged as a new sub-discipline of ecology to study ecosystems and ecological communities around the globe that are changing at an unprecedented rate because of human activities (IPCC 2013). The responses of ecosystems and communities are complex, non-linear, and driven by feedbacks across local, regional, and global scales (Heffernan et al. 2014). These characteristics necessitate novel approaches for making predictions about how systems may change to improve both our understanding of ecological phenomena as well as inform resource management.

Forecasting is a tool that can be used for understanding and predicting macrosystems dynamics. To anticipate and prepare for increased variability in populations, communities, and ecosystems, there is a pressing need to know the future state of ecological systems across space and time (Dietze et al. 2018). Ecological forecasting is an emerging approach which provides an estimate of the future state of an ecological system with uncertainty, allowing society to prepare for changes in important ecosystem services. Ecological forecasts are a powerful test of the scientific method because ecologists make a hypothesis of how an ecological system works; embed their hypothesis in a model; use the model to make a forecast of future conditions; and then when observations become available, assess the accuracy of their forecast, which indicates if their hypothesis is supported or needs to be updated. This process of iteratively updating forecasts with ecological data is called data assimilation, which can greatly improve forecast accuracy over time. Macrosystems ecologists are increasingly using ecological forecasts to predict how ecosystems are changing over space and time (Dietze and Lynch 2019).

In this module, students will generate an ecological forecast for a NEON site and explore how to use ecological data to improve forecast accuracy. This module will introduce students to the concept of data assimilation within an ecological forecast; how data assimilation can be used to improve forecast accuracy; how the level of uncertainty and temporal frequency of observations affects forecast output; and how data assimilation can affect decision-making using ecological forecasts.

Workflow for this module

Shiny app version, for either in-person or virtual instruction:

1. Instructor chooses method for accessing the Shiny app (Regardless of which option you pick, all module activities are the same!):
2. In any internet browser, go to: https://macrosystemseddie.shinyapps.io/module7/
   1. This option works well if there are not too many simultaneous users (<50) 
   2. The app generally does not take a long time to load but requires consistent internet access
   3. It is important to remind students that they need to save their work as they go, because this webpage will time out after 15 idle minutes. It is frustrating for students to lose their progress, so a good rule of thumb is to get them to save their progress after completing each objective
3. The most stable option for large classes is downloading the app and running locally, see instructions at: https://github.com/MacrosystemsEDDIE/module7
   1. Once the app is downloaded and installed (which requires an internet connection), the app can be run offline locally on students' computers
   2. This step requires R and RStudio to be downloaded on a student's computer, which may be challenging if a student does not have much R experience (but this could be done prior to instruction by an instructor on a shared computer lab)
   3. If you are teaching the module to a large class and/or have unstable internet, this is the best option
4. Give students their handout ahead of time to read over prior to class or ask students to download the handout from the module Shiny app page when they arrive to class. The module is set up for students to complete discussion questions in the student handout (a Microsoft Word document) as they navigate through the R Shiny app activities. As they navigate through the app, students will be prompted to answer questions in their handout, as well as download plots that they generate within the app and copy-paste them into their handout. The handout can be submitted to the instructor at the end of the module for potential grading. 
5. Instructor gives a PowerPoint presentation that introduces ecological forecasting, data assimilation, and the forecast model students will be using (~30 mins).
6. After the presentation, the students divide into pairs. Each pair selects their own NEON site and visualizes their site's data (Activity A Objectives 1 and 2). The two students within a pair each build their own different models for predicting chlorophyll-a (Activity A Objective 3), and generate forecasts with uncertainty using their fitted models (Activity A Objective 4). For virtual instruction, we recommend putting two pairs together (n=4 students) in separate breakout rooms during this activity so the two pairs can compare results.
7. The instructor can ask students to wait until all students are finished with Activity A and then they will all begin Activity B together. For virtual instruction, this would entail having the students come back to the main room for a short check-in.
8. In Activity B, the students work in their pairs to generate forecasts with and without data assimilation (Activity B Objective 5) as well as forecasts that assimilate data with different amounts of observation uncertainty (Activity B Objective 6) and at different frequencies (e.g., daily vs. weekly; Activity B Objective 7). Students should compare their forecasts with their partners and with students working on different lakes and work together to answer questions embedded throughout this activity about how the forecasts are affected by assimilating data with different levels of observation uncertainty and at different frequencies.
9. In Activity C, student pairs complete a management scenario individually and discuss with their partner how the forecasts with different methods of data assimilation provided in the scenario affected their management decisions (Activity C Objective 8). 

RMarkdown version, for either in-person or virtual instruction:

1. Prior to class, the instructor chooses a method for students to access and run the RMarkdown module (Regardless of which option you pick, all module activities are the same!)
   1. To access a version of the module which asks students to read and revise code to complete module activities, navigate to: https://github.com/MacrosystemsEDDIE/module7_R 
      1. This version is recommended for students and instructors with prior R coding experience
      2. To run the module, students will need R and RStudio downloaded on their computers
      3. Students can run the RMarkdown version of the module by either:
         1. downloading a zip file of the code from GitHub (easiest option) or 
         2. creating a GitHub account, forking the GitHub repository, and creating an RProject for the repository (advanced option)
2. In class, the instructor gives a brief PowerPoint presentation that introduces ecological forecasting, sources of forecast uncertainty, and the different models students will be using (~30 mins).
3. After the presentation, the students divide into pairs to work through the RMarkdown. For virtual instruction, we recommend putting two pairs together (n=4 students) in separate breakout rooms during this activity so the two pairs can compare their work.
4. We recommend regular check-ins with students as they work through the code, and no later than at the end of Activity A (the last task in Activity A is Objective 4. Generate a one-day-ahead forecast with uncertainty in the RMarkdown). The instructor can ask students to wait until all students are finished with Objective 4 and then they will all begin Activity B, Objective 5 together. For virtual instruction, this would entail having the students come back to the main room for a short check-in.
5. In Activity B, the students work in their pairs to generate forecasts with and without data assimilation (Activity B Objective 5) as well as forecasts that assimilate data with different amounts of observation uncertainty (Activity B Objective 6) and at different frequencies (e.g., daily vs. weekly; Activity B Objective 7). Students may compare their forecasts with their partners and work together to answer questions embedded throughout this activity.
6. In Activity C, student pairs complete a management scenario individually and discuss with their partner how the forecasts with different methods of data assimilation provided in the scenario affected their management decisions (Activity C Objective 8). Students may also complete an independent coding activity to fit a model and calculate uncertainty for a different water quality variable (Objective 9) and determine the optimal frequency of data assimilation for forecasts of their chosen water quality variable (Objective 10).

Teaching Materials:

• R shiny app: https://macrosystemseddie.shinyapps.io/module7/
  • To run the R Shiny app locally on your own computer, please see the instructions on the GitHub page: https://github.com/MacrosystemsEDDIE/module7
• RMarkdown GitHub repository: https://github.com/MacrosystemsEDDIE/module7_R 
  • This repository includes a README file with instructions about how to run the module
• Student_handout_Module7.docx (Microsoft Word 2007 (.docx) 1.8MB Mar7 24) - Handout with questions for students to complete
• Instructor_Manual_Module7.docx (Microsoft Word 2007 (.docx) 5.3MB Mar7 24) - Instructor manual and troubleshooting for the module
• Instructor_PowerPoint_Module7.pptx (PowerPoint 2007 (.pptx) 15.2MB Mar7 24) - PowerPoint presentation to introduce core concepts & module activities 
  • Getting_Started_With_Shiny_Module7.pptx (PowerPoint 2007 (.pptx) 7.4MB Mar7 24) - Additional PowerPoint slides that provide a basic orientation to using an R Shiny app

Important Note to Instructors:

The R Shiny app used in this module is continually being updated, so these module instructions will periodically change to account for changes in the code. If you have any questions or have other feedback about this module, please contact the module developers (see "We'd love your feedback" below).

We highly recommend that instructors familiarize themselves with the Shiny app prior to the lesson. This will enable you to be more prepared to answer questions related to certain areas of the app's functionalities.

• Activity A: Students identify patterns in chlorophyll-a data; describe the structure of the simple forecast model they fit and how well this model fits their data, and interpret and assess forecasts with uncertainty of future lake chlorophyll-a.
• Activity B: Students describe the effect of data assimilation on their forecasts and compare forecasts generated while assimilating data with low and high observation uncertainty. Students assess the effect of assimilating data at different temporal frequencies (e.g., daily, weekly) on forecast accuracy using time series figures and calculated metrics of forecast performance.
• Activity C: Students use their understanding of data assimilation to provide recommendations for development of a forecasting system as part of a management scenario.

Optional pre-class readings and videos:

Articles:

• Recommended for this module: Niu S, Luo Y, Dietze MC, Keenan TF, Shi Z, Li J, Iii FSC. 2014. The role of data assimilation in predictive ecology. Ecosphere 5: 1–16.https://doi.org/10.1890/ES13-00273.1. (Optional pre-class discussion questions associated with this paper can be found in the student handout, which is downloaded from the Shiny app).
• Silver, N. 2012. The Signal and the Noise: Why so many Predictions Fail – but some Don't. Penguin Books.
• Dietze, M., & Lynch, H. (2019). Forecasting a bright future for ecology.Frontiers in Ecology and the Environment,17(1), 3. https://doi.org/10.1002/fee.1994
• Dietze, M. C., Fox, A., Beck-Johnson, L. M., Betancourt, J. L., Hooten, M. B., Jarnevich, C. S., Keitt, T. H., Kenney, M. A., Laney, C. M., Larsen, L. G., Loescher, H. W., Lunch, C. K., Pijanowski, B. C., Randerson, J. T., Read, E. K., Tredennick, A. T., Vargas, R., Weathers, K. C., & White, E. P. (2018). Iterative near-term ecological forecasting: Needs, opportunities, and challenges. Proceedings of the National Academy of Sciences, 115(7), 1424–1432. https://doi.org/10.1073/pnas.1710231115

Videos:

• NEON's Ecological Forecast: The Science of Predicting Ecosystems
• Fundamentals of Ecological Forecasting Series
  • Why Forecast?
  • Forecast Analysis Cycle
  • Ensemble Kalman Filter

We'd love your feedback!

We frequently update this module to reflect improvements to the code, new teaching materials and relevant readings, and student activities. Your feedback is incredibly valuable to us and will guide future module development within the Macrosystems EDDIE project. Please let us know any suggestions for improvement or other comments about the module by sending an email to MacrosystemsEDDIE@gmail.com or filling out the form at the following link: https://serc.carleton.edu/eddie/macrosystems/faculty_feedback.

Module authorship contributions: CCC and RQT conceived the idea of this module and acquired funding for this project. MEL, TNM, CCC, and RQT developed the learning objectives and website text. MEL developed RMarkdown activities with feedback from CCC and RQT. MEL and TNM developed the module activities and code for the module with feedback from CCC and RQT. MEL and CCC developed and led module testing and collection and analysis of student assessment data. MEL developed the student handout, instructor powerpoint, and instructor manual with feedback from TNM, CCC and RQT. MEL, CCC, and RQT worked with instructors of the module and integrated feedback into improving the module.