Tool-Call Dependency Structure is Linearly Decodable in LLM Agent Residual Streams

2026-05-24

AI Summary

This study investigates whether large language model (LLM) agents encode the dependency structure among tool calls within their internal representations during execution. By applying low-capacity edge probing to the residual stream of Qwen3-32B and integrating counterfactual perturbations—distinguishing between value corruption and structural disruption—with activation patching and a multi-hop interaction benchmark, this work presents the first structural probing of LLM tool-call dependency graphs. Results demonstrate that the model linearly decodes a non-positional, propagative directed dependency graph, significantly outperforming random and positional baselines. Notably, this signal vanishes in single-step planning settings, confirming its specific association with multi-hop reasoning and revealing the LLM’s intrinsic capacity to represent abstract task topologies.

Abstract

Tool-using LLM agents produce trajectories whose calls form a directed dependency graph: earlier tool outputs supply arguments to later calls. Whether this execution structure is represented inside the model is unknown; prior structural probes have targeted static code or chain-of-thought text, not an agent's run-time call graph. A low-capacity edge probe on the residual stream of Qwen3-32B decodes the tool-call dependency graph well above both a Hewitt--Liang random-label control and a positional baseline. A counterfactual contrast between value corruption and structural perturbation indicates the signal tracks abstract topology rather than identifier values, and replicates under an independent, non-substring oracle. The non-positional component replicates on three further interactive multi-hop benchmarks and attenuates as call order alone becomes a sufficient proxy for dependency, vanishing in single-shot planning. Per-layer activation patching shifts the probe at a later, non-patched boundary, evidence that the representation propagates rather than passively reads out, though the realised tool call does not move. To our knowledge this is the first structural probe of an LLM agent's runtime tool-call dependency graph. Our claims concern representation, not behavioural control, and span two model families and one primary domain.

Problem

• tool-call dependency
• LLM agent
• structural probe
• residual stream
• runtime representation

Innovation

• tool-call dependency
• structural probing
• residual stream
• LLM agents
• linear decodability