Distributed Sparse Interventions in Language Models

2026-07-08

AI Summary

Existing intervention methods for language models predominantly rely on global linear directions, which fail to capture neuron-level nonlinear effects and cross-layer interactions, thereby limiting fine-grained control over model behavior. This work proposes Distributed Sparse Intervention (DSI), a method that performs sparse, structured causal interventions at the neuron level. DSI is the first to systematically model nonlinear interactions among neurons across layers and introduces set-theoretic operations to analyze the sparsity and composability of task representations. Experiments demonstrate that by intervening on only 0.01% of critical neurons, DSI can precisely activate desired behaviors across multiple tasks, significantly outperforming current linear intervention approaches while offering both high efficiency and strong interpretability.

Abstract

Language models perform a wide range of tasks at varying levels of abstraction with the capacity to flexibly infer tasks from context, execute multiple tasks simultaneously, and select among competing tasks. To study the role of model components in task behaviour, their causal influence can be investigated through interventions. Prior work on model steering has largely focused on interventions along global directions in activation space, modeling task representations as approximately linear and additive. By studying interventions at the neuron level, we find substantial, neuron-specific nonlinear effects on model outputs that are not captured by current steering approaches. We introduce Distributed Sparse Interventions (DSI), an intervention approach that considers nonlinearities and interactions between neurons across layers to identify sparse sets of neurons that elicit task-relevant computations. Across a range of tasks, we demonstrate that DSI can activate task behaviour in instruction-tuned language models by localising and intervening on as few as 0.01% of neurons, highlighting the effectiveness of sparse, distributed interventions in the neuron basis. Additionally, adopting a set-based perspective enables computations over the identified neuron sets, offering insights into the roles of individual neurons by analysing their effects across tasks. Through sparse interventions, DSI enables fine-grained control over model behaviour, localisation of task-relevant neuron sets, and furthers our understanding of task composition.

Problem

• distributed sparse interventions
• nonlinear effects
• neuron-level interventions
• task-relevant neurons
• language model steering

Innovation

• Distributed Sparse Interventions
• neuron-level intervention
• nonlinear effects
• task composition
• sparse neuron sets