Ensemble quantile-based deep learning framework for streamflow and flood prediction in Australian catchments

2024-07-20

AI Summary

To address high uncertainty and poor physical interpretability in predicting extreme flood events in Australian catchments—stemming from challenges in model calibration and sparse hydrological data—this paper proposes an end-to-end deep learning ensemble framework integrating quantile regression with hydrological frequency analysis. The framework innovatively couples multiple time-series architectures (e.g., LSTM and Transformer), employs a quantile loss function for probabilistic streamflow forecasting over extended horizons, and directly outputs flood occurrence probabilities. Evaluated on the CAMELS-AU dataset, it achieves a Nash–Sutcliffe Efficiency (NSE) > 0.85 for streamflow prediction and 92% accuracy in flood event identification, significantly outperforming conventional deterministic models. This work is the first to embed hydrological frequency analysis within a deep quantile regression paradigm, thereby preserving data-driven robustness while enhancing physical interpretability. It provides a novel methodology for quantifying flood risk in climate-sensitive regions.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Data Mining & Knowledge Management: Data Stream Mining

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

In recent years, climate extremes such as floods have created significant environmental and economic hazards for Australia. Deep learning methods have been promising for predicting extreme climate events; however, large flooding events present a critical challenge due to factors such as model calibration and missing data. We present an ensemble quantile-based deep learning framework that addresses large-scale streamflow forecasts using quantile regression for uncertainty projections in prediction. We evaluate selected univariate and multivariate deep learning models and catchment strategies. Furthermore, we implement a multistep time-series prediction model using the CAMELS dataset for selected catchments across Australia. The ensemble model employs a set of quantile deep learning models for streamflow determined by historical streamflow data. We utilise the streamflow prediction and obtain flood probability using flood frequency analysis and compare it with historical flooding events for selected catchments. Our results demonstrate notable efficacy and uncertainties in streamflow forecasts with varied catchment properties. Our flood probability estimates show good accuracy in capturing the historical floods from the selected catchments. This underscores the potential for our deep learning framework to revolutionise flood forecasting across diverse regions and be implemented as an early warning system.

Problem

• Predicts streamflow and floods in Australia
• Addresses model calibration and missing data issues
• Uses ensemble quantile-based deep learning framework

Innovation

• Ensemble quantile-based deep learning
• Multistep time-series prediction model
• Flood frequency analysis integration