Search papers, labs, and topics across Lattice.
This paper introduces RosettaSim, a novel framework for long-term traffic simulation that leverages the inductive biases and statistical properties of large-scale sequence models, particularly large language models (LLMs). By structuring agent interactions into an autoregressive stream, RosettaSim achieves high accuracy in both short-term and long-horizon simulations, while also addressing the challenge of agent correspondence over extended rollouts through a Retrieval-based Traffic Evaluation (RTE) method. Evaluations on the Waymo Open Sim Agent Challenge show that RosettaSim not only sets a new state-of-the-art in simulation performance but also enhances fidelity metrics significantly compared to existing methods.
RosettaSim redefines traffic simulation by harnessing LLMs, achieving superior long-term fidelity and accuracy in agent interactions.
Interactive traffic simulation is a vital world model for autonomous driving. A central challenge in long-horizon simulation is modeling sustained multi-agent interactions, which is further exacerbated by dynamic token cardinality as agents continuously enter and exit the scene. In this work, we propose that the solution lies in the synergy between the architectural inductive biases and statistical priors of large-scale sequence models, e.g., Large Language Models (LLMs). Our probing experiments reveal that the transferability of attention mechanisms and the distributional consistency between motion tokens and natural language enable small-scale, heavily frozen LLMs to rapidly adapt to traffic modeling. Building on this insight, we introduce RosettaSim, a unified framework that projects scene topology, agent states, and spawning intents into a structured autoregressive stream with variable length, achieving both strong short-term accuracy and stable long-horizon simulation fidelity. Furthermore, evaluating extended rollouts presents yet another hurdle, as one-to-one agent correspondence inevitably fades over time. To address this, we introduce Retrieval-based Traffic Evaluation (RTE), which retrieves semantically similar real-world scenarios as context-aware reference anchors. Experiments on the Waymo Open Sim Agent Challenge (WOSAC) demonstrate that RosettaSim achieves state-of-the-art performance in both short- and long-term simulation. Furthermore, RTE exhibits a stronger correlation with standard metrics ($r=0.83$) than existing approaches ($r=0.74$), indicating improved alignment with long-horizon simulation fidelity.
