FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers

2026-05-16

AI Summary

This work addresses a fundamental limitation in existing activation intervention methods, which erroneously treat the activation space as Euclidean while ignoring the intrinsic non-Euclidean geometry induced by the model’s output distribution. By leveraging the Fisher information metric associated with the softmax layer and pulling it back to intermediate layers via the Jacobian of subsequent layers, the authors derive—for the first time—a closed-form expression for the optimal intervention direction that requires neither manifold fitting nor data-dependent estimation. This approach explicitly captures the true geometric structure of the activation space and reveals that prior methods implicitly rely on approximate metrics. Experiments on GPT-2 demonstrate that the proposed pullback-based intervention consistently outperforms Euclidean baselines across three verb morphology tasks and four network layers, achieving target probabilities while reducing KL divergence from non-target distributions by 1.3–2.5× compared to gradient ascent and by 1.5× relative to CAA.

Abstract

Activation steering methods modify intermediate representations of language models to control output behavior, but universally assume the activation space is Euclidean. We show this assumption fails drastically: the local geometry induced by the model's own output behavior -- the Fisher information metric of the softmax layer, pulled back through the Jacobian of subsequent layers -- deviates from the Euclidean metric by over 97% in relative spectral norm on GPT-2, with an effective dimensionality of only 2--17% of the ambient space. From this pullback Fisher metric, we derive a closed-form steering equation that identifies the minimum-distortion direction for any target concept, yielding a closed-form optimal direction at each point that can be applied iteratively without manifold fitting or data-driven geometry estimation. We call the resulting framework FishBack. The metric admits a layer-wise recursive decomposition, which reveals that existing methods -- CAA, ActAdd, ITI, and others -- each implicitly adopt a particular approximate metric, and that their performance gaps are quantitatively predicted by a single spectral diagnostic: the ratio of their implicit metric's cost to the Fisher-optimal cost. On GPT-2, iterative pullback steering consistently outperforms all Euclidean baselines across three verb-morphology concepts and four layers, with off-target KL reductions of $1.3\times$--$2.5\times$ relative to Euclidean gradient ascent and $1.5\times$ relative to CAA at matched concept probability.

Problem

• activation steering
• Fisher information metric
• non-Euclidean geometry
• transformer models
• representation geometry

Innovation

• Pullback Fisher metric
• Activation steering
• Optimal direction
• Non-Euclidean geometry
• Transformer interpretability