Evaluating Deterministic and Ensemble Streamflow Estimates

HydroLearn Evaluating Deterministic and Ensemble Streamflow Estimates How Good Is Your Forecast? Evaluating Deterministic and Ensemble Streamflow Models View Course Problem Statement Hydrologic models are routinely used to forecast streamflow for flood warning, reservoir operation, and water-supply planning. Yet the same model can look great on one event and miss the next, and a colorful hydrograph plot can hide important errors. Worse, modern forecast systems do not return a single number any more. They return a cloud of plausible streamflows, an ensemble , and that cloud needs its own kind of verification. Module Overview This module gives early-career hydrologists a compact, hands-on introduction to how forecasters actually judge whether a streamflow model is doing its job . It moves you from continuous metrics (RMSE, NSE, KGE, PBIAS) and Moriasi performance ratings, through categorical event verification (POD, FAR, POFD, CSI, ROC), to probabilistic ensemble verification (Brier score, BSS, reliability diagrams, rank histograms, CRPS). At every step you read a real hydrograph, score it, and decide whether the result is fit for an operational decision. The hands-on activities use the HyMOD conceptual rainfall-runoff model on the Leaf River catchment through a sequence of four Python notebooks, runnable directly in Google Colab. Topics Covered Deterministic evaluation metrics (RMSE, NSE, KGE, PBIAS); Moriasi performance ratings; hydrograph diagnostics; flow-duration curves; ensemble forecasting; contingency tables; categorical scores (POD, FAR, POFD, CSI); ROC curves and performance diagrams; Brier score and Brier skill score; reliability diagrams; rank histograms; Continuous Ranked Probability Score (CRPS); the HyMOD conceptual model. Prerequisites Before starting this module, learners should have: Basic familiarity with hydrologic concepts (precipitation, runoff, streamflow). Working knowledge of Python (numpy, pandas, matplotlib), enough to run a Jupyter notebook end-to-end. No experience with HyMOD or with ensemble systems is required. Comfort reading a hydrograph. Learning Objectives At the end of this module, learners will be able to: Choose, compute, and interpret the right deterministic metric (NSE, KGE, PBIAS, RMSE, peak error) for a given forecasting use case. Read a hydrograph and a flow-duration curve to diagnose what kind of error a model is making. Build a 2×2 contingency table and compute POD, FAR, POFD, and CSI for a flood-warning forecast. Read a ROC curve, a reliability diagram, and a rank histogram to judge whether an ensemble is sharp, reliable, and well-dispersed. Compute and interpret the Brier score, Brier skill score, and CRPS, and recommend whether an ensemble forecast is fit for a stated operational decision. This is accomplished through a series of short readings on fundamental concepts, accompanied by two hands-on Python learning activities and a capstone authentic task. Suggested Implementation This module is broken down into four sections with small units. Each section is self-contained and can be exercised individually. Total estimated effort is 1 to 1.5 hours, self paced. Section Estimated time Section 1, Introduction 5 min Section 2, Deterministic evaluation 25 to 30 min (includes Learning Activity 1) Section 3, Ensemble evaluation 30 to 35 min (includes Learning Activity 2) Section 4, Authentic task 10 to 15 min Course Authors Mohamed Abdelkader IIHR Hydroscience and Engineering, University of Iowa mohamed-abdelkader@uiowa.edu Humberto Vergara IIHR Hydroscience and Engineering, University of Iowa Target Audience Early-career hydrologists and operational forecasters at National Meteorological and Hydrological Services (NMHSs) who want to confidently evaluate streamflow models for operational decisions. Tools Needed A computer with internet access, a modern browser, and a Google account to run the accompanying Jupyter notebooks in Google Colab (no local install required). Basic working knowledge of Python is helpful. Expected Effort About 1 hour 15 minutes total (1 to 1.5 hour range). Self paced. Course Sharing and Adaptation This course is available for export by clicking the "Export Link" at the top right of this page. You will need a HydroLearn instructor studio account to do this. You will first need to sign up for a hydrolearn.org account, then register as an instructor by clicking 'studio.hydrolearn' and requesting course creation permissions. Recommended Citation Abdelkader, M., Vergara, H. (2026). How Good Is Your Forecast? Evaluating Deterministic and Ensemble Streamflow Models. WMO Capacity-Building Activity. University of Iowa. Acknowledgments Module developed for the World Meteorological Organization (WMO) capacity-building activity, built around the HyMOD ensemble-verification notebook series by Humberto Vergara and Mohamed Abdelkader (University of Iowa). References to the supporting literature are listed in the appendix of each section.