Search papers, labs, and topics across Lattice.
AnyScene is introduced, a novel occupancy-centric framework for generating controllable driving scene videos from BEV layouts. It uses a Spatial-Temporal Occupancy Diffusion Transformer to generate semantic occupancy sequences autoregressively, enabling fine-grained control and long-horizon generation. A Geometry-Grounded View Expansion module then synthesizes temporally consistent multi-view videos from the occupancy, supporting flexible camera configurations.
Unlock scalable autonomous driving simulation with AnyScene, a framework that generates controllable, high-fidelity driving scenes from arbitrary BEV layouts and camera configurations.
Generating high-fidelity and controllable synthetic data is critical for advancing end-to-end autonomous driving, particularly for addressing the long tail of rare safety-critical scenarios. Existing occupancy-guided methods typically rely on shallow conditioning mechanisms and reference-frame-dependent video synthesis, which limits fine-grained controllability from arbitrary BEV layouts and restricts their applicability for scalable simulation. In this paper, we propose AnyScene, a unified occupancy-centric framework for driving scene generation. AnyScene generates semantic occupancy sequences from BEV layouts through a Spatial-Temporal Occupancy Diffusion Transformer that jointly tokenizes BEV and occupancy features in an autoregressive manner. This design enables precise controllability from cross-dataset and user-defined BEV inputs while naturally supporting long-horizon generation. Building upon the generated occupancy, a Geometry-Grounded View Expansion module treats occupancy as the canonical spatial representation and synthesizes temporally consistent multi-view driving videos in a reference-free and autoregressive fashion, supporting flexible camera configurations at inference time. Extensive experiments demonstrate that AnyScene achieves state-of-the-art performance in both occupancy and video generation. It exhibits strong generalization to unseen and customized layouts, and provides measurable benefits for downstream tasks such as sparse-view 3D reconstruction.
Tsinghua AI