Search papers, labs, and topics across Lattice.
This paper introduces SCIT, a causal protocol designed to analyze the mechanisms behind latent chain-of-thought models by constructing counterfactuals and identifying the transformer objects responsible for computation. The findings reveal that counterfactual arithmetic predominantly transfers through value-cache suffix trajectories rather than other mechanisms, with evidence supporting a late-value-suffix mechanism in specific model checkpoints. Additionally, the study uncovers distinct carrier-regime shifts in different model sizes, providing insights into the operational differences between arithmetic-like and competent cells in GPT-2 architectures.
Counterfactual arithmetic in latent models primarily travels through value-cache suffixes, challenging assumptions about hidden states and keys.
Latent chain-of-thought models move intermediate reasoning from emitted text into continuous states, improving compactness but hiding the causal object. We introduce SCIT, the Suffix Cache Interchange Test, a causal protocol that constructs exact source-recipient counterfactuals, patches declared cache segments, and identifies which transformer object carries the counterfactual computation. SCIT combines sufficiency tests with K/V component splits, hidden-state controls, semantic source controls, decoded validation, and matched corruption. On CODI-GPT2 and a Sim-CoT-style GPT-2 reproduction, counterfactual arithmetic transfers primarily through value-cache suffix trajectories rather than hidden states, keys, reusable answer slots, or single-token triggers. Complete sufficiency-and-necessity evidence for the late-value-suffix mechanism holds for the main CODI-GPT2 checkpoint; the Sim-CoT-style checkpoint shows the same sufficiency and decoded-control pattern but insufficient matched-corruption evidence for a necessity call. Beyond these local arithmetic cells, SCIT reveals carrier-regime shifts: arithmetic-like GPT-2/1B cells preserve latent-tail value/KV transfer, whereas competent 8B and repaired non-arithmetic cells route through prompt-prefix or full-cache K/V; boundary cells receive no mechanism call. SCIT therefore contributes a cache-level diagnostic, a checkpoint-specific GPT-2 arithmetic mechanism, and a competence-gated carrier map rather than a universal latent-tail claim.
