UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering

2026-05-28

AI Summary

This work addresses the limitations of existing activation-based steering methods for large language models, which struggle with fine-grained concepts and compositional constraints while relying on fixed directions or task-specific modules. The authors propose UniSteer, the first unified conditional activation flow-matching framework that enables general and flexible intervention on frozen LLMs. By learning a text-conditioned velocity field in the residual stream activation space, UniSteer integrates text-conditioned flow matching, partial transport and regeneration mechanisms in activation space, and a reconstruction-energy-based classification strategy—eliminating the need for separate modules per behavior type. Experiments demonstrate that UniSteer efficiently supports diverse tasks—including behavioral control, truthfulness guidance, fine-grained concept manipulation, multi-constraint instruction following, and activation-space classification—across three target models within a single, cohesive framework.

Abstract

Activation-based control steers large language models (LLMs) by intervening on their internal representations during inference, and has emerged as an effective paradigm for controlling behaviors such as persona and style. However, existing methods often rely on fixed steering directions or task-specific intervention modules, making them difficult to adapt to fine-grained concepts and compositional constraints. We propose UniSteer, a text-guided activation flow matching model that learns a conditional distribution over residual-stream activations from natural-language conditions. Instead of fitting a separate intervention for each target behavior, UniSteer learns a universal conditional velocity field in activation space. At inference time, UniSteer performs flow inversion by partially transporting a source activation toward a latent state and regenerating it under a target textual condition before injecting it back into the frozen LLM. The same conditional model supports activation-space classification by selecting the textual label with the lowest reconstruction energy. Experiments on three target LLMs show that UniSteer provides a unified interface across behavioral control, truthfulness steering, fine-grained concept steering, multi-constraint instruction following, and activation-space classification.

Problem

• activation-based control
• large language models
• fine-grained concept steering
• compositional constraints
• behavioral control

Innovation

• activation-based control
• flow matching
• text-guided steering
• conditional velocity field
• large language models