Improved probabilistic regression using diffusion models

2025-10-06

AI Summary

Existing diffusion models struggle to effectively model predictive distributions in probabilistic regression, lack adaptive uncertainty quantification, and exhibit limited generalization. This paper proposes a generic Diffusion Probabilistic Regression (DPR) framework that nonparametrically learns the full conditional distribution of the response variable, supporting diverse regression tasks from low- to high-dimensional settings. Its core innovation is a learnable noise-distribution parameterization that explicitly models and adaptively captures predictive uncertainty throughout the diffusion process; we systematically analyze the trade-off between calibration and generalization under varying noise structures. Distribution fitting and uncertainty estimation are jointly optimized via denoising training. Experiments on multiple benchmark regression tasks demonstrate that DPR significantly outperforms state-of-the-art methods, yielding more accurate and better-calibrated uncertainty estimates—validating its effectiveness and robustness on complex data.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Probabilistic regression models the entire predictive distribution of a response variable, offering richer insights than classical point estimates and directly allowing for uncertainty quantification. While diffusion-based generative models have shown remarkable success in generating complex, high-dimensional data, their usage in general regression tasks often lacks uncertainty-related evaluation and remains limited to domain-specific applications. We propose a novel diffusion-based framework for probabilistic regression that learns predictive distributions in a nonparametric way. More specifically, we propose to model the full distribution of the diffusion noise, enabling adaptation to diverse tasks and enhanced uncertainty quantification. We investigate different noise parameterizations, analyze their trade-offs, and evaluate our framework across a broad range of regression tasks, covering low- and high-dimensional settings. For several experiments, our approach shows superior performance against existing baselines, while delivering calibrated uncertainty estimates, demonstrating its versatility as a tool for probabilistic prediction.

Problem

• Developing diffusion models for probabilistic regression tasks
• Enhancing uncertainty quantification in predictive distributions
• Creating versatile nonparametric framework for diverse regression settings

Innovation

• Nonparametric diffusion framework models predictive distributions
• Full distribution modeling of diffusion noise enhances uncertainty
• Versatile noise parameterizations adapt to diverse regression tasks