Search papers, labs, and topics across Lattice.
This paper investigates whether LLMs represent the dependency graph of tool calls within their residual streams during agentic execution. Using a linear edge probe on Qwen1.5-32B's residual stream, the authors successfully decode the tool-call dependency graph, outperforming random and positional baselines. Counterfactual analysis reveals the probe captures abstract topology rather than identifier values, suggesting a genuine understanding of dependencies.
LLMs encode tool-call dependency graphs in their residual streams, revealing a structured understanding of agentic execution beyond simple sequence prediction.
Tool-using LLM agents produce trajectories whose calls form a directed dependency graph: earlier tool outputs supply arguments to later calls. Whether this execution structure is represented inside the model is unknown; prior structural probes have targeted static code or chain-of-thought text, not an agent's run-time call graph. A low-capacity edge probe on the residual stream of Qwen3-32B decodes the tool-call dependency graph well above both a Hewitt--Liang random-label control and a positional baseline. A counterfactual contrast between value corruption and structural perturbation indicates the signal tracks abstract topology rather than identifier values, and replicates under an independent, non-substring oracle. The non-positional component replicates on three further interactive multi-hop benchmarks and attenuates as call order alone becomes a sufficient proxy for dependency, vanishing in single-shot planning. Per-layer activation patching shifts the probe at a later, non-patched boundary, evidence that the representation propagates rather than passively reads out, though the realised tool call does not move. To our knowledge this is the first structural probe of an LLM agent's runtime tool-call dependency graph. Our claims concern representation, not behavioural control, and span two model families and one primary domain.