Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejection

2026-03-06

AI Summary

This work addresses the vulnerability of large language models to over-manipulation by existing activation steering methods when following complex instructions, which often degrades both generation quality and task accuracy. To mitigate this issue, the authors propose DIRECTER, a novel approach that achieves dynamic, rejection-based steering without requiring any additional data. By performing lightweight attention sensitivity analysis to assess output plausibility, DIRECTER adaptively modulates the strength of activation steering—implemented via KV cache scaling—during decoding. Evaluated across multiple instruction-following benchmarks, the method significantly improves task accuracy (by up to 6.5%) while preserving high generation quality and task fidelity.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Large Language Models (LLMs), despite advances in instruction tuning, often fail to follow complex user instructions. Activation steering techniques aim to mitigate this by manipulating model internals, but have a potential risk of oversteering, where excessive emphasis on the instruction degrades task accuracy and overall text quality. To address this, we introduce DIRECTER (Dynamic rejection steering), a novel steering method that dynamically modulates steering strength by scaling the KV cache without extra dataset. DIRECTER couples steering with a plausibility-guided decoding loop, which adaptively adjusts steering strength at each step by comparing the steered output distribution to the original. If the steered output is deemed implausible, steering strength is progressively weakened. This strength modulation is guided by a lightweight, one-time attention sensitivity analysis that ranks layers by their influence on model representations. Extensive evaluations show that DIRECTER significantly enhances instruction-following capabilities across diverse benchmarks, improving accuracy by up to 6.5% over baselines without the common trade-offs in generation quality or task fidelity. The proposed dynamic, plausibility-guided control during activation steering further demonstrates its potential as a general mechanism for mitigating oversteering that is compatible with existing baselines.

Problem

• instruction following
• large language models
• activation steering
• oversteering
• complex instructions

Innovation

• activation steering
• dynamic rejection
• instruction following
• KV cache modulation
• plausibility-guided decoding