Understanding Uncertainty in Ecological Forecasts

This module was developed by Moore, T. N., Carey, C.C. and Thomas, R. Q. 13 October 2021. Macrosystems EDDIE: Understanding Uncertainty in Ecological Forecasts. Macrosystems EDDIE Module 6, Version 1. http://module6.macrosystemseddie.org. Module development was supported by NSF grants DEB-1926050 and DBI-1933016.

Author Profile

Summary

Ecological forecasting is a tool that can be used for understanding and predicting changes in populations, communities, and ecosystems. Ecological forecasting is an emerging approach which provides an estimate of the future state of an ecological system with uncertainty, allowing society to prepare for changes in important ecosystem services. Forecast uncertainty is derived from multiple sources, including model parameters, driver data, among others. Knowing the uncertainty associated with a forecast enables forecast users to evaluate the forecast and make more informed decisions. Ecological forecasters develop and update forecasts using the iterative forecasting cycle, in which they make a hypothesis of how an ecological system works; embed their hypothesis in a model; and use the model to make a forecast of future conditions and quantify forecast uncertainty. There are a number of approaches that forecasters can use to reduce uncertainty, which will be explored in this module.

Used this activity? Share your experiences and modifications

Learning Goals

By the end of this module, students will be able to:

  • Define ecological forecast uncertainty
  • Explore the contributions of different sources of uncertainty (e.g., model parameters, model driver data) to total forecast uncertainty
  • Understand how multiple sources of uncertainty are quantified
  • Identify ways in which uncertainty can be reduced within an ecological forecast
  • Describe how forecast horizon affects forecast uncertainty
  • Explain the importance of specifying uncertainty in ecological forecasts for forecast users and decision support

Context for Use

This entire module can be completed in one 2 to 3-hour lab period or two 60-minute lecture periods for introductory undergraduate students in Ecology, Environmental Science, Ecological Modelling, and Quantitative Ecology classes. This module can be partnered with Module 5 "Introduction to Ecological Forecasting" which provides students with a more general introduction to ecological forecasting. Otherwise, there are slides included which be integrated if Module 5 is not taught before.We found that teaching this module in one longer lab section with short breaks was more conducive for introductory students than multiple 1-hour lecture periods.

Description and Teaching Materials

Quick overview of the activities in this module

We are currently revising the instructor manual, student handout, and instructor PowerPoint. These will be posted on this site when they are completed (expected December 2021 for winter teaching).

  • Activity A: Students build different models to simulate water temperature for their chosen NEON site.
  • Activity B: Students generate multiple forecasts of water temperature with different sources of uncertainty and examine how uncertainty propagation differs.
  • Activity C: Students then quantify and partition the uncertainty for their forecasts with different models and compare uncertainty across lake sites.

Why macrosystems ecology and ecological forecasting?

Macrosystems ecology is the study of ecological dynamics at multiple interacting spatial and temporal scales (e.g., Heffernan et al. 2014). For example, global climate change can interact with local land-use activities to control how an ecosystem changes over the next decades. Macrosystems ecology recently emerged as a new sub-discipline of ecology to study ecosystems and ecological communities around the globe that are changing at an unprecedented rate because of human activities (IPCC 2013). The responses of ecosystems and communities are complex, non-linear, and driven by feedbacks across local, regional, and global scales (Heffernan et al. 2014). These characteristics necessitate novel approaches for making predictions about how systems may change to improve both our understanding of ecological phenomena as well as inform resource management.

Forecasting is a tool that can be used for understanding and predicting macrosystems dynamics. To anticipate and prepare for increased variability in populations, communities, and ecosystems, there is a pressing need to know the future state of ecological systems across space and time (Dietze et al., 2018). Ecological forecasting is an emerging approach which provides an estimate of the future state of an ecological system with uncertainty, allowing society to prepare for changes in important ecosystem services. Ecological forecasts are a powerful test of the scientific method because ecologists make a hypothesis of how an ecological system works; embed their hypothesis in a model; use the model to make a forecast of future conditions; and then when observations become available, assess the accuracy of their forecast, which indicates if their hypothesis is supported or needs to be updated. Forecasts that specify uncertainty will be most useful for aiding decision-making. Consequently, macrosystems ecologists are increasingly using ecological forecasts to predict how ecosystems are changing over space and time.

In this module, students will generate an ecological forecast for a NEON site and quantify the different sources of uncertainty within their forecast. This module will introduce students to the concept of uncertainty within an ecological forecast; where uncertainty in a forecast comes from; how uncertainty can be quantified within a forecast; and how uncertainty can be managed.

 

Workflow for this module:

  1. Give students their handout ahead of time to read over prior to class, or distribute handouts when they arrive to class. For virtual instruction, we recommend uploading the handout to a learning management system (e.g., Blackboard, Canvas, Moodle) for students to fill in questions as they proceed through the materials.
  2. Instructor gives a brief PowerPoint presentation that introduces ecological forecasting, forecast uncertainty, the water temperature model students will be adapting in the module, and the sources of forecast uncertainty, and how forecast uncertainty can be reduced.
  3. After the presentation, the students divide into pairs. Each pair selects a NEON lake site and builds models to predict water temperature. For virtual instruction, we recommend putting two pairs together (n=4 students) into separate Zoom breakout rooms.
  4. Following Activity A, the instructor leads a short discussion comparing the different models using the discussion questions embedded in the Shiny app.
  5. The instructor then introduces what will be done in Activity B on the Shiny app. For virtual instruction, this would entail having the students come back to the main Zoom room for a short check-in.
  6. The students work in their pairs and step through Objectives 6-8. If students complete these objectives early, they can answer some of the discussion questions embedded in the Shiny app. For virtual instruction, we recommend putting the two pairs back into the same Zoom breakout rooms.
  7. Following Activity B, the instructor leads a short discussion comparing the different forecasts and why the uncertainty differs between the models and at different time horizons using the discussion questions embedded in the Shiny app.
  8. For Activity C, each student in their pairs selects two water temperature models and partitions the uncertainty for the same NEON site. In pairs, the students explore why their different models have different contributions of uncertainty sources.
  9. The students work together in a group to present the results from their two sites and different models and discuss why their forecasts and forecast uncertainty contributions are similar or different among the different NEON sites and models. 

Teaching Materials:

  • Currently in revision, will be available in December 2021.

Teaching Notes and Tips

Important Note to Instructors:

The R Shiny app was built using R version 4.1.0. All dependent R packages used in this module are continually being updated, so these module instructions will periodically change to account for changes in the code.

If you have any questions or any problems with this module, please reach out to us at MacrosystemsEDDIE@gmail.com.

Assessment

  • Activity A: Students are introduced to forecast models and different uncertainty sources and then quantify a model's performance and describe its structure.
  • Activity B: Students generate forecasts with different sources of uncertainty and answer questions on how to reduce forecast uncertainty
  • Activity C: Students compare forecast uncertainty among NEON sites; partition their forecast's uncertainty into its individual sources; and compare the contribution of different forecast sources over different forecast horizons.

References and Resources

Optional pre-class readings and videos:

Articles:

  • Silver, N. (2012) Chapter 6: How to drown in three feet of water. Pages 176-203 in The Signal and the Noise: Why so many Predictions Fail – but some Don't. Penguin Books.
  • Dietze, M., & Lynch, H. (2019). Forecasting a bright future for ecology. Frontiers in Ecology and the Environment, 17(1), 3. https://doi.org/10.1002/fee.1994
  • Dietze, M. C., Fox, A., Beck-Johnson, L. M., Betancourt, J. L., Hooten, M. B., Jarnevich, C. S., Keitt, T. H., Kenney, M. A., Laney, C. M., Larsen, L. G., Loescher, H. W., Lunch, C. K., Pijanowski, B. C., Randerson, J. T., Read, E. K., Tredennick, A. T., Vargas, R., Weathers, K. C., & White, E. P. (2018). Iterative near-term ecological forecasting: Needs, opportunities, and challenges. Proceedings of the National Academy of Sciences, 115(7), 1424–1432. https://doi.org/10.1073/pnas.1710231115

Videos: