Tackling the Accuracy-Interpretability Trade-off in a Hierarchy of Machine Learning Models for the Prediction of Extreme Heatwaves

2024-10-01

arXiv.org

AI Summary

Climate forecasting faces a fundamental trade-off between model accuracy and physical interpretability. Method: This study develops a multi-level heatwave prediction framework—comprising Gaussian approximation (GA), intrinsically interpretable neural networks (IINN), scattering networks (ScatNet), and convolutional neural networks (CNN)—applied to extreme heatwaves in France. Contribution/Results: We present the first climate-domain validation demonstrating that ScatNet achieves both high predictive accuracy and intrinsic scale-aware interpretability: its heatwave probability prediction attains an AUC of 0.89, matching CNN performance (+0.03) and significantly outperforming GA and IINN. Crucially, ScatNet enables direct identification of dominant spatiotemporal scales and key atmospheric circulation patterns without requiring post-hoc explanation methods—thereby overcoming the “accuracy–interpretability paradox” inherent in black-box deep learning. This work establishes a novel methodological pathway toward trustworthy, physics-informed AI for climate science.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Interpretability, Explainability, and Transparency
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

When performing predictions that use Machine Learning (ML), we are mainly interested in performance and interpretability. This generates a natural trade-off, where complex models generally have higher skills but are harder to explain and thus trust. Interpretability is particularly important in the climate community, where we aim at gaining a physical understanding of the underlying phenomena. Even more so when the prediction concerns extreme weather events with high impact on society. In this paper, we perform probabilistic forecasts of extreme heatwaves over France, using a hierarchy of increasingly complex ML models, which allows us to find the best compromise between accuracy and interpretability. More precisely, we use models that range from a global Gaussian Approximation (GA) to deep Convolutional Neural Networks (CNNs), with the intermediate steps of a simple Intrinsically Interpretable Neural Network (IINN) and a model using the Scattering Transform (ScatNet). Our findings reveal that CNNs provide higher accuracy, but their black-box nature severely limits interpretability, even when using state-of-the-art Explainable Artificial Intelligence (XAI) tools. In contrast, ScatNet achieves similar performance to CNNs while providing greater transparency, identifying key scales and patterns in the data that drive predictions. This study underscores the potential of interpretability in ML models for climate science, demonstrating that simpler models can rival the performance of their more complex counterparts, all the while being much easier to understand. This gained interpretability is crucial for building trust in model predictions and uncovering new scientific insights, ultimately advancing our understanding and management of extreme weather events.

Problem

• Balancing accuracy and interpretability in ML for heatwave prediction
• Comparing ML models from Gaussian Approximation to CNNs
• Identifying transparent models like ScatNet for climate science

Innovation

• Hierarchy of ML models for heatwave prediction
• ScatNet balances accuracy and interpretability
• Interpretable models rival complex CNN performance