Short Horizons and Sparse Concepts: a Mathematical View of the Readout in the J-lens

2026-08-26

AI Summary

本文通过数学方法探讨J-lens从语言模型中读取可表述表示的原理，揭示其因果结构，并提出改进策略以增强J-lens读取正确中间概念的能力。

Abstract

The Jacobian lens (J-lens) has been proposed as a way to read verbalizable representations from language models. However, its principle and meaning lack a detailed and theoretical discussion. We provide a mathematical view of this interpretation and of its assumed causal structure. Besides treating the J-lens as a heuristic probe, we further regard it as a first-order causal transfer operator from intermediate activations to expected future readouts. We study the Jacobian matrix as the optimal local linear approximation of the downstream mapping, analyze its global approximation behavior and bias, and identify its mathematical meaning as an expectation over anticipated future readouts. Further analysis of the Jacobian energy distribution reveals that its causal geometry is highly sparse. The energy decays with depth, concentrates in an extremely small proportion, and decomposes into diagonal pathways and specific critical positions. This decomposition further resolves the expectation of the J-lens over future outputs into short-horizon and sparse concept predictions, providing a more intuitive attribution and explanation for the ability of the J-lens to visualize concepts during the thinking process. Based on the theory, we propose a simple but effective improvement strategy and decoupling method for the J-lens, which significantly enhances the ability of the J-lens to read out correct intermediate concepts.

Problem

• Jacobian lens
• language models
• causal structure
• approximation
• sparse concepts

Innovation

• Jacobian lens
• causal transfer operator
• energy distribution
• sparse concepts
• short-horizon predictions