Density-Ratio Losses for Post-Hoc Learning to Defer

2026-05-19

AI Summary

This work proposes a flexible and robust post-hoc learning-to-defer mechanism that operates without retraining the underlying model. By introducing an idealized expert distribution, the deferral decision is formulated as the density ratio between the model’s predictive distribution and this ideal distribution. Leveraging the equivalence between density ratio estimation and class probability estimation, the authors construct a tunable scoring function with an adjustable threshold to dynamically control the deferral rate. This approach is the first to formalize post-hoc learning-to-defer as a density ratio estimation problem, unifying Chow’s rule and expert-comparison methods while revealing its intrinsic connection to anomaly detection. Experiments demonstrate that the method matches or outperforms existing baselines across multiple datasets, exhibits strong robustness across diverse scenarios, and enables flexible adjustment of the deferral rate without any retraining.

Abstract

We study post-hoc Learning to Defer (L2D) through the lens of ideal distributions: divergence-regularized reweightings of the data distribution under which a model attains low loss. We define deferral via the density-ratio between a model's and an expert's ideals. Using the reduction from density-ratio estimation to class-probability estimation, we derive the DR CPE losses for post-hoc L2D scorers. Deferral decisions are then made by thresholding the scorer, allowing deferral rates to be adjusted without retraining. For KL-based ideal distributions, our deferral rules recovers Chow's rule under the original distribution and a connection to an expert-tilted Bayes posterior -- which incorporates the expert's performance -- depending on if the ideal distributions are joint or marginal distributions. Experimentally, our approach is competitive compared to common baselines and more robust across dataset settings. More broadly, our results cast post-hoc L2D as density-ratio learning between ideal distributions, bridging Chow-style rules, expert comparison, and elucidating connections to related learning settings including anomaly detection.

Problem

• Learning to Defer
• Density-Ratio Estimation
• Post-hoc Learning
• Expert Deferral
• Ideal Distributions

Innovation

• density-ratio estimation
• learning to defer
• ideal distributions
• post-hoc calibration
• Chow's rule