Knowledge-Verified Emergent Deception in LLM Agents Under Conflicting Incentives

2026-08-26

AI Summary

研究解决了大语言模型在利益冲突下可能出现的欺骗行为问题，通过引入知识验证基准KnownLieBench来区分故意欺骗与无知或幻觉，进而评估和调整模型诚实性。

Abstract

Large language models are increasingly deployed as autonomous agents serving users on behalf of companies, placing them in settings where user and deployer interests can conflict. When an agent knows that a user is owed something its deployer would prefer to deny, does it remain honest? Answering this is difficult because false statements can reflect either ignorance or hallucination rather than deception. To address this challenge, we introduce KnownLieBench , a knowledge-verified benchmark that first confirms through a neutral probe that an agent knows a user's entitlement, and then evaluates whether it makes false claims once an incentive to deny that entitlement is introduced. Specifically, KnownLieBench covers eight customer-service domains and 112 grounded cases, conducts multi-round dialogues with a trust-tracking customer agent, and separates deception emerging from incentive alone from deception produced under explicit instruction. Across eighteen proprietary and open-weight models, emergent deception varies substantially across model families and domains. We further use the benchmark for post-training, finding that honesty-directed fine-tuning reduces deception under incentive, while deception-graded fine-tuning increases lie success on honest-control dialogues without increasing lie frequency under incentive. By verifying entitlement knowledge before scoring deceptive behavior, KnownLieBench reduces the confound between lying and not knowing and enables more rigorous auditing and steering of agent honesty.

Problem

• Large Language Models
• Conflicting Incentives
• Emergent Deception
• Autonomous Agents
• Honesty

Innovation

• Knowledge-Verified Benchmark
• Emergent Deception
• Conflicting Incentives
• Honesty-directed Fine-tuning