Uncertainty Quantification for Prior-Data Fitted Networks using Martingale Posteriors

2025-05-16

AI Summary

While Prototype-based Forecasting Networks (PFNs) excel at prediction on small- to medium-scale datasets, they lack principled uncertainty quantification for key predictive quantities such as means and quantiles. Method: This paper introduces Martingale posterior theory into PFNs for the first time, constructing a provably convergent Bayesian posterior distribution. The proposed framework enables uncertainty quantification without retraining by integrating Martingale posterior sampling with the PFN architecture and establishing statistical reliability via asymptotic convergence analysis. Contribution/Results: Experiments across multiple synthetic and real-world tabular datasets demonstrate that the method yields well-calibrated uncertainty estimates. It is the first uncertainty quantification approach for PFNs with rigorous theoretical guarantees—ensuring validity in statistical inference—and significantly enhances PFNs’ credibility and practical utility in downstream decision-making tasks.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Programming
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Prior-data fitted networks (PFNs) have emerged as promising foundation models for prediction from tabular data sets, achieving state-of-the-art performance on small to moderate data sizes without tuning. While PFNs are motivated by Bayesian ideas, they do not provide any uncertainty quantification for predictive means, quantiles, or similar quantities. We propose a principled and efficient sampling procedure to construct Bayesian posteriors for such estimates based on Martingale posteriors, and prove its convergence. Several simulated and real-world data examples showcase the uncertainty quantification of our method in inference applications.

Problem

• PFNs lack uncertainty quantification for predictions
• Propose Bayesian posteriors via Martingale posteriors
• Validate method with simulated and real data

Innovation

• Martingale posteriors for Bayesian uncertainty quantification
• Efficient sampling procedure for predictive estimates
• Convergence-proven method for real-world data