Beyond Linear Steering: Unified Multi-Attribute Control for Language Models

2025-05-30

AI Summary

Controlling multiple behavioral attributes simultaneously in large language model (LLM) inference remains challenging due to restrictive linear intervention assumptions, inter-attribute interference, and the need to store separate direction vectors per attribute. Method: We propose K-Steering—a non-linear, fine-tuning-free framework that replaces additive linear interventions with a single multi-label classifier. It dynamically generates composite intervention directions in real time by leveraging gradients of hidden-layer activations. Contributions/Results: We introduce two novel benchmarks—ToneBank and DebateMix—evaluated via both activation-based classifiers and LLM-based adjudication. On three mainstream LLMs, K-Steering significantly outperforms linear baselines in multi-attribute joint control, achieving state-of-the-art performance. The framework demonstrates strong generalization, precise controllability, and practical engineering viability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Intelligent Robots: Behavior Learning & Control

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Controlling multiple behavioral attributes in large language models (LLMs) at inference time is a challenging problem due to interference between attributes and the limitations of linear steering methods, which assume additive behavior in activation space and require per-attribute tuning. We introduce K-Steering, a unified and flexible approach that trains a single non-linear multi-label classifier on hidden activations and computes intervention directions via gradients at inference time. This avoids linearity assumptions, removes the need for storing and tuning separate attribute vectors, and allows dynamic composition of behaviors without retraining. To evaluate our method, we propose two new benchmarks, ToneBank and DebateMix, targeting compositional behavioral control. Empirical results across 3 model families, validated by both activation-based classifiers and LLM-based judges, demonstrate that K-Steering outperforms strong baselines in accurately steering multiple behaviors.

Problem

• Control multiple behavioral attributes in LLMs
• Overcome interference between attributes and linear limitations
• Enable dynamic behavior composition without retraining

Innovation

• Non-linear multi-label classifier for hidden activations
• Gradient-based intervention directions at inference
• Dynamic behavior composition without retraining