Calibration-Aware Uncertainty Cascades for Efficient Heterogeneous Model Collaboration

2026-09-10

AI Summary

提出Calibration-Aware Uncertainty Cascades（CAUC）框架，通过独立校准各模型置信度并使用验证数据选择部署策略，以解决异构模型协作中预测性能与推理成本的平衡问题。

Abstract

Heterogeneous model collaboration seeks to exploit the complementary strengths of different models to balance predictive performance and inference cost. Existing approaches typically rely either on trained routers, which tie routing decisions to a fixed task and model pool, or on raw-confidence cascades, whose thresholds lack consistent reliability semantics across heterogeneous models. Consequently, these approaches adapt poorly to changing model pools and deployment budgets. We propose Calibration-Aware Uncertainty Cascades (CAUC), a simple post-hoc framework that independently calibrates each model's confidence and selects deployment policies using validation data. The resulting calibrated confidence scores establish a common reliability scale for accepting an early prediction, invoking a stronger model, or selectively combining model outputs. This unified decision criterion decouples deployment policies from any particular model pool or operating budget. We further show theoretically that calibration gives confidence thresholds an explicit selective-risk interpretation, whereas uncalibrated scores offer no comparable reliability guarantee. Extensive experiments demonstrate that, across six language benchmarks, CAUC achieves an average relative accuracy improvement of 1.9% over strong-model-only inference while avoiding approximately 47% of strong-model calls. On image classification benchmarks, it maintains or improves predictive performance while reducing measured GFLOPs by up to 57%.

Problem

• Heterogeneous model collaboration
• calibration
• uncertainty
• inference cost
• predictive performance

Innovation

• Calibration-Aware Uncertainty Cascades
• heterogeneous model collaboration
• confidence calibration
• deployment policy
• selective risk