Search papers, labs, and topics across Lattice.
Investigating the causal gap between feature representation and execution for dynamically inferred traits, this work tracks where transformers represent and act on a conversational partner's expertise across multi-turn dialogues. Linear probing shows that inferred partner expertise peaks in early layers and decays to near-chance before the network's midpoint, whereas counterfactual activation patching reveals that only post-midpoint injections propagate to late-layer readouts鈥攁n order-of-magnitude causal disconnect. Unlike explicitly stated attributes that persist throughout the residual stream, dynamically inferred relational features decouple decodability from causal efficacy, sharply bounding where persona-steering interventions must occur.
Transformers encode an inferred partner's expertise early in the network, but interventions at peak decodability produce virtually zero downstream effect until patched past the midpoint.
A transformer can make an attribute linearly decodable in its residual stream at a depth where that attribute does not yet influence the output. This gap between where information is readable and where it is used has been shown for attributes stated directly in the input. We ask whether it also holds for an attribute the model must infer gradually over a conversation, namely how expert its dialogue partner is. Using ExpertCollab, a corpus of multi-turn research-planning dialogues between model-played personas at four expertise levels, we find that partner expertise is most decodable in the early layers and falls to near chance before the midpoint of the network. Counterfactual patching shows that injecting the expertise difference at the layer of peak decodability barely changes a fixed late-layer readout, whereas the same difference injected past the midpoint propagates almost completely, a separation of more than an order of magnitude. A content-matched random control and a probe-free diagnostic place the transition at the same early layer, and a statically specified control attribute stays decodable throughout. An inferred relational attribute is therefore represented well before it becomes causally active, which bounds where any attempt to read out or steer partner-conditioned behavior must intervene. We use one model on a synthetic corpus as an initial demonstration.
