Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To overcome the quadratic memory bottleneck that limits Algorithm Distillation in long-horizon environments, this work develops Recurrent Algorithm Distillation (RAD) to decouple effective history length from Transformer context size. Standard in-context reinforcement learning requires prohibitively large context windows to track multi-episode policy updates, restricting its practical deployment. Across diverse benchmarks, RAD matches the asymptotic performance of full-context Algorithm Distillation while operating over a fixed-size latent memory buffer with significantly smaller context footprints.
Long-horizon in-context RL no longer requires exploding context windows—compressing past learning trajectories into a fixed-size latent buffer matches full Algorithm Distillation performance at a fraction of the memory cost.
Algorithm Distillation (AD) has demonstrated the remarkable ability of Transformers to perform in-context reinforcement learning without explicit weight updates. However, capturing long-term learning progress necessitates expansive context windows, which incur prohibitive memory costs and limit scalability in complex, long-horizon tasks. To address this bottleneck, we propose Recurrent Algorithm Distillation (RAD). RAD employs a dual-component architecture: a Compression Transformer that distills extended interaction histories into compact latent tokens, and an AD Transformer that auto-regressively generates actions using a hybrid context of these compressed memories and recent transitions. By maintaining a fixed-size latent buffer, RAD decouples the effective history length from computational complexity, functionally providing the model with a long-horizon memory. Empirical evaluations across diverse environments demonstrate that RAD matches the asymptotic performance of standard AD with significantly reduced context window sizes, offering a scalable solution for efficient in-context decision-making.