Conformity Breaks Conformal Prediction

2026-09-03

AI Summary

研究解决了多智能体系统中一致性预测失效的问题，通过分析模型在同伴压力下的评分机制变化，并揭示了标准校正方法对此问题无效的原因。

Abstract

A conformal certificate can be valid when an LLM answers alone and invalid when the same LLM sees peers that unanimously assert a wrong answer. The question is unchanged; the model's score for the correct answer changes. We call this a score-mechanism shift: clean calibration certifies how the model scores answers alone, but not how it scores them under peer pressure. We show that this shift silently breaks conformal prediction in multi-agent LLM systems. Across open-weight models and multiple-choice QA tasks, coverage falls from a calibrated 90% to 74% under unanimous-wrong peers at the standard alpha = 0.10 operating point. The average hides a sharper failure: by targeting the low-confidence items the certificate still covers, an attacker nearly halves coverage on that subgroup, from 87% to 47%, while the monitored average remains much higher. The failure also reaches the decision layer: a system that should escalate when uncertain can instead become confident enough to act on the attacker's wrong answer. Standard conformal fixes do not solve the problem, because the question distribution has not changed; the model's scoring behavior has.

Problem

• conformal prediction
• score-mechanism shift
• multi-agent LLM systems

Innovation

• score-mechanism shift
• conformal prediction failure
• multi-agent LLM systems