Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
This study investigates the assumptions underlying stage-replay diagnostics in a Qwen2.5-derived system by comparing the effects of retained live cache versus one-shot prefill of identical integer tokens. The findings reveal that while BF16 precision leads to discrepancies in suffixes and correctness labels, FP32 maintains consistency without disagreement, suggesting that numerical precision significantly influences model behavior. Ultimately, the research establishes that divergent continuations can be replicated through bidirectional transplantation of key/value layers, indicating that cache can effectively carry divergent trajectories without needing to preserve live-state fidelity.
Bidirectional transplantation of key/value layers allows for repeatable exact-token replay, challenging assumptions about live-state fidelity in model behavior.
Stage-replay diagnostics reconstruct intermediate token prefixes and treat fresh-prefill continuation as continuation from the decoder state that originally reached the prefix. We audit that assumption at a whole reasoning-stage boundary in a Qwen2.5-derived system. A matched 200-item experiment compares retained live cache with one-shot prefill of identical integer tokens and places an exact replica on both sides. In BF16, replicas remain exact while the constructions differ on 166 suffixes and 20 correctness labels; the accuracy difference is only one point (paired 95% CI [-3.5, +5.5]). A fixed-prefix 2x2 holds all 200 token states constant while crossing construction and precision. The BF16 disagreements recur, whereas FP32 produces no decoded disagreement (95% Wilson upper bound 1.88%). A prospective bridge makes token-by-token incremental and retained live caches bit-exact on 12/12 rows; an all-200 saved-ledger audit reproduces every retained trajectory and comparison fingerprint. Bidirectional transplantation of all 48 key/value layers makes every tested divergent continuation follow its cache donor, both on a selected set at the primary checkpoint (24/24) and an outcome-blind replication at a later checkpoint (43/43). Exact-token replay can therefore be repeatable without preserving live-state fidelity. On the tested states, boundary K/V cache is a causally sufficient carrier of the divergent trajectory, while numerical precision moderates its behavioral expression.