Understanding Uncertainty in Ecological Forecasts

This module was developed by Moore, T. N., Carey, C.C. and Thomas, R. Q. 13 October 2021. Macrosystems EDDIE: Understanding Uncertainty in Ecological Forecasts. Macrosystems EDDIE Module 6, Version 1. http://module6.macrosystemseddie.org. Module development was supported by NSF grants DEB-1926050 and DBI-1933016.

Author Profile

Summary

Ecological forecasting is a tool that can be used for understanding and predicting changes in populations, communities, and ecosystems. Ecological forecasting is an emerging approach which provides an estimate of the future state of an ecological system with uncertainty, allowing society to prepare for changes in important ecosystem services. Forecast uncertainty is derived from multiple sources, including model parameters and driver data, among others. Knowing the uncertainty associated with a forecast enables forecast users to evaluate the forecast and make more informed decisions. Ecological forecasters develop and update forecasts using the iterative forecasting cycle, in which they make a hypothesis of how an ecological system works; embed their hypothesis in a model; and use the model to make a forecast of future conditions and quantify forecast uncertainty. There are a number of approaches that forecasters can use to reduce uncertainty, which will be explored in this module.

Used this activity? Share your experiences and modifications

Learning Goals

By the end of this module, students will be able to:

  • Define ecological forecast uncertainty
  • Explore the contributions of different sources of uncertainty (e.g., model parameters, model driver data) to total forecast uncertainty
  • Understand how multiple sources of uncertainty are quantified
  • Identify ways in which uncertainty can be reduced within an ecological forecast
  • Describe how forecast horizon affects forecast uncertainty
  • Explain the importance of specifying uncertainty in ecological forecasts for forecast users and decision support

Context for Use

This entire module can be completed in one 2 to 3-hour lab period or two 75-minute lecture periods for introductory undergraduate students in Ecology, Environmental Science, Ecological Modelling, and Quantitative Ecology classes. This module can be coupled with other Macrosystems EDDIE ecological forecasting modules: Module 5 "Introduction to Ecological Forecasting"; Module 7 "Using Data to Improve Ecological Forecasts"; or Module 8 "Using Ecological Forecasts to Guide Decision-Making". We found that teaching this module in one longer lab section with short breaks was more conducive for introductory students than multiple 1-hour lecture periods.

Description and Teaching Materials

Quick overview of the activities in this module

See the instructor manual, provided below, for a step-by-step guide for carrying out this module. A student handout describing Activities A, B, and C, and an instructor PowerPoint are also provided.

  • Activity A: Students build different models to simulate water temperature for their chosen NEON site.
  • Activity B: Students generate multiple forecasts of water temperature with different sources of uncertainty and examine how uncertainty propagation differs.
  • Activity C: Students quantify and partition the uncertainty for their forecasts with different models and compare uncertainty across lake sites.

Why macrosystems ecology and ecological forecasting?

Macrosystems ecology is the study of ecological dynamics at multiple interacting spatial and temporal scales (e.g., Heffernan et al. 2014). For example, global climate change can interact with local land-use activities to control how an ecosystem changes over the next decades. Macrosystems ecology recently emerged as a new sub-discipline of ecology to study ecosystems and ecological communities around the globe that are changing at an unprecedented rate because of human activities (IPCC 2013). The responses of ecosystems and communities are complex, non-linear, and driven by feedbacks across local, regional, and global scales (Heffernan et al. 2014). These characteristics necessitate novel approaches for making predictions about how systems may change to improve both our understanding of ecological phenomena as well as inform resource management.

Forecasting is a tool that can be used for understanding and predicting macrosystems dynamics. To anticipate and prepare for increased variability in populations, communities, and ecosystems, there is a pressing need to know the future state of ecological systems across space and time (Dietze et al., 2018). Ecological forecasting is an emerging approach which provides an estimate of the future state of an ecological system with uncertainty, allowing society to prepare for changes in important ecosystem services. Ecological forecasts are a powerful test of the scientific method because ecologists make a hypothesis of how an ecological system works; embed their hypothesis in a model; use the model to make a forecast of future conditions; and then when observations become available, assess the accuracy of their forecast, which indicates if their hypothesis is supported or needs to be updated. Forecasts that specify uncertainty will be most useful for aiding decision-making. Consequently, macrosystems ecologists are increasingly using ecological forecasts to predict how ecosystems are changing over space and time.

In this module, students will generate an ecological forecast for a NEON site and quantify the different sources of uncertainty within their forecast. This module will introduce students to the concept of uncertainty within an ecological forecast; where uncertainty in a forecast comes from; how uncertainty can be quantified within a forecast; and how uncertainty can be managed.

Workflow for this module:

  1. Give students their handout ahead of time to read over prior to class or distribute handouts when they arrive to class. The R Shiny app is set up for students to complete discussion questions as they navigate through the module activities. Thus, students could answer questions 1-3 prior to the start of instruction and can save their progress, which will allow them to return at a different time. The questions can be saved and downloaded as a Microsoft Word file at the end of the module, which could be submitted to their instructor for potential grading.
  2. Instructor gives a brief PowerPoint presentation that introduces ecological forecasting, sources of forecast uncertainty, and the different models they will be using (~30 mins).
  3. After the presentation, the students divide into pairs. Each pair selects their own NEON site and visualizes their site's data (Site selection Objectives 1 and 2). The two students within a pair each build their own different models for predicting water temperature (Activity A Objective 3), explore model parameter distributions (Activity A Objective 4), and improve upon their original models to build a suite of simple models for forecasting (Activity A Objective 5). For virtual instruction, we recommend putting two pairs together (n=4 students) into separate breakout rooms during this activity so the two pairs can compare results.
  4. The instructor can instruct students to wait until all students are finished Activity A and then they will all begin Activity B together. For virtual instruction, this would entail having the students come back to the main room for a short check-in.
  5. In Activity B, the students work in their pairs to generate different forecasts which include different sources of uncertainty (Activity B Objectives 6-9). Students first compare their forecasts with their partners and work together to hypothesize why the different models are affected differently by the different sources of uncertainty (Activity B Summary).
  6. In Activity C, student pairs generate forecasts including all sources of uncertainty and compare how the different models forecast different ranges and have different contributions of uncertainty (Activity C Objective 10). They then complete the management scenario individually and discuss with their partner how the uncertainty visualization affected their management decision (Activity C Objective 11).

Teaching Materials:

Teaching Notes and Tips

Important Note to Instructors:

The R Shiny app used in this module is continually being updated, so these module instructions will periodically change to account for changes in the code. If you have any questions or have other feedback about this module, please contact the module developers at MacrosystemsEDDIE@gmail.com.

We highly recommend that instructors familiarize themselves with the Shiny app prior to the lesson. This will enable you to be more prepared to answer questions related to certain areas of the app's functionalities.

Assessment

  • Activity A: Students are introduced to forecast models and different uncertainty sources and then quantify a model's performance and describe its structure.
  • Activity B: Students generate forecasts with different sources of uncertainty and answer questions on how to reduce forecast uncertainty
  • Activity C: Students compare forecast uncertainty among NEON sites; partition their forecast's uncertainty into its individual sources; and compare the contribution of different forecast sources over different forecast horizons.

References and Resources

Optional pre-class readings and videos:

Articles:

  • Silver, N. (2012) Chapter 6: How to drown in three feet of water. Pages 176-203 in The Signal and the Noise: Why so many Predictions Fail – but some Don't. Penguin Books.
  • Dietze, M., & Lynch, H. (2019). Forecasting a bright future for ecology. Frontiers in Ecology and the Environment, 17(1), 3. https://doi.org/10.1002/fee.1994
  • Dietze, M. C., Fox, A., Beck-Johnson, L. M., Betancourt, J. L., Hooten, M. B., Jarnevich, C. S., Keitt, T. H., Kenney, M. A., Laney, C. M., Larsen, L. G., Loescher, H. W., Lunch, C. K., Pijanowski, B. C., Randerson, J. T., Read, E. K., Tredennick, A. T., Vargas, R., Weathers, K. C., & White, E. P. (2018). Iterative near-term ecological forecasting: Needs, opportunities, and challenges. Proceedings of the National Academy of Sciences, 115(7), 1424–1432. https://doi.org/10.1073/pnas.1710231115

Videos: