Symmetric Aggregation of Conformity Scores for Efficient Uncertainty Sets

2025-12-07

AI Summary

To address the low efficiency of uncertainty aggregation and redundant prediction sets in multi-model conformal prediction, this paper proposes Symmetric Aggregation Conformal Prediction (SACP). SACP calibrates each model’s nonconformity scores via e-value transformation—the first application of e-values in this context—and employs symmetric aggregation functions (e.g., product or arithmetic mean) to achieve theoretically guaranteed robust combination, yielding more compact yet statistically valid prediction sets. Its core innovation lies in integrating e-value theory with symmetric aggregation, balancing theoretical rigor with data-driven adaptability. Extensive experiments across multiple benchmark datasets demonstrate that SACP significantly reduces prediction set width—achieving an average improvement of 12.7%—while strictly maintaining the nominal coverage probability. It consistently outperforms existing model-aggregation approaches in both efficiency and statistical reliability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Access to multiple predictive models trained for the same task, whether in regression or classification, is increasingly common in many applications. Aggregating their predictive uncertainties to produce reliable and efficient uncertainty quantification is therefore a critical but still underexplored challenge, especially within the framework of conformal prediction (CP). While CP methods can generate individual prediction sets from each model, combining them into a single, more informative set remains a challenging problem. To address this, we propose SACP (Symmetric Aggregated Conformal Prediction), a novel method that aggregates nonconformity scores from multiple predictors. SACP transforms these scores into e-values and combines them using any symmetric aggregation function. This flexible design enables a robust, data-driven framework for selecting aggregation strategies that yield sharper prediction sets. We also provide theoretical insights that help justify the validity and performance of the SACP approach. Extensive experiments on diverse datasets show that SACP consistently improves efficiency and often outperforms state-of-the-art model aggregation baselines.

Problem

• Aggregating predictive uncertainties from multiple models
• Combining individual prediction sets into a single informative set
• Improving efficiency and reliability of uncertainty quantification

Innovation

• Aggregates nonconformity scores from multiple predictors
• Transforms scores into e-values for symmetric aggregation
• Enables data-driven selection for sharper prediction sets