Structure-Adaptive Conformal Inference for Large-Scale Out-of-Distribution Testing

2026-05-25

AI Summary

This work addresses the challenge of structured out-of-distribution (OOD) detection in high-stakes machine learning settings, where existing conformal methods struggle to leverage auxiliary structural information such as temporal, spatial, or group dependencies. The authors propose Structured Conformal q-values (SCQ) and Pseudo-score-guided Transductive Auto Model Selection (P-TAMS), which, for the first time, integrate individual test evidence with structural patterns within a pairwise exchangeability framework—thereby overcoming the limitations imposed by traditional joint exchangeability assumptions. The approach achieves rigorous error rate control under finite samples while simultaneously enhancing detection power and interpretability. Extensive experiments on both synthetic and real-world structured OOD scenarios demonstrate substantial improvements over baseline methods, with effective control of the false discovery rate.

Abstract

This paper addresses structured out-of-distribution (OOD) testing in high-stakes machine learning applications. Traditional conformal methods rely on joint exchangeability, making it difficult to incorporate auxiliary information such as spatiotemporal or grouping structures. To overcome this limitation, we propose the structure-adaptive conformal q-value (SCQ), a significance index that integrates individual test evidence with structural patterns. We also develop pseudo-score-guided transductive automated model selection (P-TAMS), which adapts conformalized model selection to structured OOD testing across a toolbox of candidate models. Together, SCQ and P-TAMS form a unified framework under pairwise exchangeability, providing finite-sample error-rate control, improved power, and enhanced interpretability. Experiments on simulated and real data demonstrate that the proposed approach controls the false discovery rate and performs well across diverse settings.

Problem

• out-of-distribution testing
• structured data
• conformal inference
• false discovery rate
• high-stakes machine learning

Innovation

• structure-adaptive conformal inference
• out-of-distribution testing
• conformal q-value
• model selection
• false discovery rate control