Search papers, labs, and topics across Lattice.
The paper introduces T^2MLR, a novel architecture that enhances Transformer models by integrating a cached middle-layer representation from previous tokens into earlier layers, allowing for better retention of intermediate reasoning states during autoregressive decoding. This method significantly improves performance in natural language pretraining and multi-hop reasoning tasks, outperforming baseline Transformers with equivalent data and parameters. Notably, T^2MLR can be retrofitted into existing pretrained models, demonstrating substantial gains in math reasoning with minimal additional training, thereby lowering the barrier for practical implementation.
Targeted middle-layer recurrence can dramatically enhance Transformer reasoning without the need for full-layer looping, leading to superior performance in complex tasks.
Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reasoning states to persist across time. We in troduce Transformers with Temporal Middle-Layer Recurrence (T2MLR), a transformers-based latent reasoning architecture that fuses a cached middle layer representation from the previous token directly into an earlier layer of the current token position, enabling abstract intermediate computation to persist across decoding steps with little inference overhead. Across natural-language pretraining and multi-hop reasoning finetuning, T2MLR consistently outperforms data- and parameter-matched Transformer base lines. Moreover, applying recurrence to only a localized middle-layer block (as little as 20% of the network) often outperforms full-layer recurrence. Im portantly, T2MLR does not require pretraining from scratch: retrofitting the recurrent pathway into an existing pretrained 1.7B Transformer and briefly finetuning substantially improves math reasoning, lowering the barrier to practical adoption. These results suggest that effective latent reasoning in Transformers does not require looping over all layers as in previous works, but can instead emerge more strongly from targeted middle-layer recurrence.
