Search papers, labs, and topics across Lattice.
This paper introduces a fast-slow recurrent architecture for sequential modeling that interleaves rapid latent state updates with slower, self-organizing observation updates. This approach promotes the formation of stable, clustered internal representations, enabling the model to maintain coherence over extended sequences. The proposed method outperforms LSTMs, state space models, and Transformers on out-of-distribution generalization tasks in reinforcement learning and algorithmic domains.
Forget Transformers; this new recurrent architecture learns more stable representations and generalizes better out-of-distribution by interleaving fast latent updates with slower, self-organizing observation processing.
We extend the recent latent recurrent modeling to sequential input streams. By interleaving fast, recurrent latent updates with self-organizational ability between slow observation updates, our method facilitates the learning of stable internal structures that evolve alongside the input. This mechanism allows the model to maintain coherent and clustered representations over long horizons, improving out-of-distribution generalization in reinforcement learning and algorithmic tasks compared to sequential baselines such as LSTM, state space models, and Transformer variants.
