JAPAN: Joint Adaptive Prediction Areas with Normalising-Flows

2025-05-29

AI Summary

Existing residual-based conformal prediction methods suffer from geometric rigidity, overly conservative prediction sets, and confinement to the mean under multimodal distributions. This paper proposes the first density-based conformal prediction framework, innovatively leveraging normalizing flow models to estimate conditional densities and constructing a conformity score directly from the estimated density. The resulting prediction sets are shape-adaptive, non-convex, possibly disconnected, and context-sensitive—all while guaranteeing finite-sample marginal coverage. We theoretically establish its statistical efficiency advantage over residual methods. Empirically, on multivariate regression and time-series forecasting tasks, our method reduces average prediction set volume by 18–35% compared to state-of-the-art residual approaches, while achieving significantly improved calibration.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Multi-modal Vision

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Conformal prediction provides a model-agnostic framework for uncertainty quantification with finite-sample validity guarantees, making it an attractive tool for constructing reliable prediction sets. However, existing approaches commonly rely on residual-based conformity scores, which impose geometric constraints and struggle when the underlying distribution is multimodal. In particular, they tend to produce overly conservative prediction areas centred around the mean, often failing to capture the true shape of complex predictive distributions. In this work, we introduce JAPAN (Joint Adaptive Prediction Areas with Normalising-Flows), a conformal prediction framework that uses density-based conformity scores. By leveraging flow-based models, JAPAN estimates the (predictive) density and constructs prediction areas by thresholding on the estimated density scores, enabling compact, potentially disjoint, and context-adaptive regions that retain finite-sample coverage guarantees. We theoretically motivate the efficiency of JAPAN and empirically validate it across multivariate regression and forecasting tasks, demonstrating good calibration and tighter prediction areas compared to existing baselines. We also provide several emph{extensions} adding flexibility to our proposed framework.

Problem

• Overly conservative prediction areas in conformal prediction
• Difficulty capturing multimodal distributions with residual-based scores
• Need for adaptive, compact prediction regions with coverage guarantees

Innovation

• Uses density-based conformity scores
• Leverages flow-based models
• Constructs compact adaptive prediction areas