StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions

2026-09-01

AI Summary

研究通过引入双框架协议和特殊标记[STATE]，探讨并解决大型语言模型在支持和排除框架下回答选择题时的一致性问题。

Abstract

Large language models often answer the same multiple-choice question inconsistently when it is posed under support-oriented and elimination-oriented framings. We investigate whether these discrepancies arise from different internal representations induced by the two framings. We introduce a dual-framing protocol with minimally varied prompts that use either support- or elimination-oriented framing while keeping the evaluation target fixed. To probe the internal computation, we append an untrained special token, [STATE], and treat its residual-stream activation as an intervention interface. Across both models, the two framings induce separable [STATE] activations concentrated in intermediate layers. Swapping these activations between paired prompts systematically changes predictions and improves cross-framing agreement, providing intervention-based evidence that the activations are behaviorally relevant. Beyond instance-level substitution, mean-difference steering directions derived from the dual-framing contrast exhibit more bounded layer-wise responses than matched contrastive activation addition directions under the evaluated protocol.

Problem

• large language models
• multiple-choice questions
• support-elimination
• internal representations
• framing

Innovation

• dual-framing protocol
• internal representation
• [STATE] activation
• cross-framing agreement
• mean-difference steering