Cost-Aware Post-Hoc Deferral Under Calibration and Shift: An Environmental AI Case Study

📅 2026-09-07
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
研究通过EcoTrust框架解决在环境AI中选择延期策略的问题,考虑了成本、校准偏差和数据偏移等因素,使用错误风险估计器等方法优化决策。
📝 Abstract
Choosing a deferral policy for a frozen classifier requires more than ranking uncertain cases: confidence may be miscalibrated, errors have unequal costs, reviewers can err, and deployment data can leave calibration support. We study these interactions through EcoTrust, a post-hoc framework that compares automatic action with review using a six-group error-risk estimator, class-asymmetric costs, reviewer accuracy, and an optional support gate. On a Columbia River thermal-stress testbed, the learned estimator improves error-ranking area under the receiver operating characteristic curve from 0.869 to 0.889, but Chow's confidence rule has lower in-distribution cost (0.416 versus 0.567 per day). Across 12 off-the-shelf backends, learned risk and a calibration-matched, class-aware confidence estimator each beat raw Chow on six; a paired year-block bootstrap does not resolve their mean cost difference. In transfer to ten river stations, the gate flags every case and becomes an always-review fallback, attaining the lowest cost on eight stations only when review is perfect and unconstrained. These results characterize decision boundaries on one controlled task: richer risk signals do not reliably improve on calibrated confidence, and detected extrapolation does not imply transferable case-level ranking.
Problem

Research questions and friction points this paper is trying to address.

deferral policy
miscalibrated confidence
unequal costs
reviewer accuracy
calibration support
Innovation

Methods, ideas, or system contributions that make the work stand out.

Cost-Aware Deferral
Post-Hoc Framework
Error-Risk Estimator
Class-Asymmetric Costs
Reviewer Accuracy
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
H
Haoran Yu
University of Florida
L
Lifei Liu
Wichita State University
D
Danping Zhang
Nanchang Hangkong University