Search papers, labs, and topics across Lattice.
The paper introduces RAYNOVA, a geometry-free world model for autonomous driving that uses a dual-causal autoregressive framework to model spatio-temporal correlations in a unified manner. RAYNOVA employs relative Plücker-ray positional encoding to construct an isotropic spatio-temporal representation, avoiding reliance on explicit 3D geometric priors and enabling generalization to diverse camera setups. The model achieves state-of-the-art multi-view video generation on nuScenes with improved throughput and controllability, while also demonstrating generalization to novel views and camera configurations.
Ditching explicit 3D geometry, RAYNOVA achieves SOTA multi-view video generation by modeling spatio-temporal relationships directly with a dual-causal autoregressive framework and Plücker-ray positional encoding.
World foundation models aim to simulate the evolution of the real world with physically plausible behavior. Unlike prior methods that handle spatial and temporal correlations separately, we propose RAYNOVA, a geometry-free world model that employs a dual-causal autoregressive framework. It follows both scale-wise and temporal topological orders in the autoregressive process, and leverages global attention for unified 4D spatio-temporal reasoning. Different from existing works that impose strong 3D geometric priors, RAYNOVA constructs an isotropic spatio-temporal representation across views, frames, and scales based on relative Plücker-ray positional encoding, enabling robust generalization to diverse camera setups and ego motions. We further introduce a recurrent training paradigm to alleviate distribution drift in long-horizon video generation. RAYNOVA achieves state-of-the-art multi-view video generation results on nuScenes, while offering higher throughput and strong controllability under diverse input conditions, generalizing to novel views and camera configurations without explicit 3D scene representation. Our code will be released at http://yichen928.github.io/raynova.
BAIR