Search papers, labs, and topics across Lattice.
This paper introduces ECoSim, a lightweight control adaptation framework that enhances multi-modal controllability in traffic simulation models without the need for extensive retraining or large annotated datasets. By employing identity-initialized FiLM layers to modulate intermediate features, ECoSim efficiently integrates new control modalities while preserving the generative capabilities of existing models. Evaluated on the Waymo Open Sim Agents Challenge, the framework achieves robust controllability with less than 1% of the paired control data, facilitating counterfactual scenario generation and maintaining driving realism and safety.
Achieving robust controllability in traffic simulation with less than 1% of the required control data could revolutionize how we test autonomous driving systems.
Controllable traffic simulation is critical for testing autonomous driving systems, yet existing approaches often require retraining large generative models with extensive annotated data. We introduce a lightweight control adaptation framework that enables multi-modal controllability (sketch, latent behavior codes, and text) for pretrained state-of-the-art diffusion and autoregressive traffic models. By modulating intermediate features through identity-initialized FiLM layers, our method efficiently adds new control modalities while preserving the base model's generative prior. Evaluated on Waymo Open Sim Agents Challenge, our approach demonstrates strong controllability with less than 1% of the paired control data. Through context-aware condition transfer, our framework enables counterfactual scenario generation and long-tail synthesis while maintaining stable closed-loop driving realism and safety. Our framework unlocks new possibilities for controllable traffic simulation, enabling targeted scenario generation through lightweight adaptation of pretrained generative models. Project page: https://ecosim-web.github.io/
