Search papers, labs, and topics across Lattice.
This paper scales Decision Pre-trained Transformers (DPTs) to multi-domain in-context reinforcement learning by using Flow Matching to maintain the Bayesian posterior sampling interpretation of DPTs. The resulting agent, trained on hundreds of diverse tasks, generalizes better to held-out test sets compared to Algorithm Distillation (AD) baselines. The work demonstrates that DPTs can be scaled effectively, offering a competitive alternative to expert distillation for training generalist agents in ICRL.
DPTs, previously limited to simplified domains, now crush Algorithm Distillation in multi-domain in-context RL, proving they can generalize to unseen tasks.
Recent progress in in-context reinforcement learning (ICRL) has demonstrated its potential for training generalist agents that can acquire new tasks directly at inference. Algorithm Distillation (AD) pioneered this paradigm and was subsequently scaled to multi-domain settings, although its ability to generalize to unseen tasks remained limited. The Decision Pre-Trained Transformer (DPT) was introduced as an alternative, showing stronger in-context reinforcement learning abilities in simplified domains, but its scalability had not been established. In this work, we extend DPT to diverse multi-domain environments, applying Flow Matching as a natural training choice that preserves its interpretation as Bayesian posterior sampling. As a result, we obtain an agent trained across hundreds of diverse tasks that achieves clear gains in generalization to the held-out test set. This agent improves upon prior AD scaling and demonstrates stronger performance in both online and offline inference, reinforcing ICRL as a viable alternative to expert distillation for training generalist agents.
