Evaluating the Semantic Specificity of Representation Steering in Language Models

2026-08-29

AI Summary

本文提出Cross-Rule Transfer框架，评估语言模型中Localized Representation Steering方法的有效性，发现该方法仅引入全局标签偏差而非真正修复推理错误。

Abstract

Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluations can easily be fooled by superficial label overrides, creating a false impression of reasoning circuit repairs. In this work, we propose Cross-Rule Transfer (CRT), a diagnostic framework that audits representational interventions by evaluating them on rule families where the model is natively competent. Evaluating late-layer LRS for a widespread logical failure, contradiction blindness, reveals that the intervention merely injects a global label bias: applying the steering vector to rules the model already handles correctly (99.6% baseline) degrades performance to 40.4% by forcing false contradiction predictions. We support this diagnosis with four complementary controls (direct logit bias equivalence, control vector label-flipping, cross-model grafting, and early-layer steering checks), providing a rigorous methodology to distinguish genuine reasoning repairs from superficial label overrides.

Problem

• Localized Representation Steering
• Semantic Specificity
• Reasoning Pathologies
• Contradiction Blindness
• Cross-Rule Transfer

Innovation

• Cross-Rule Transfer
• Localized Representation Steering
• Contradiction Blindness
• Global Label Bias