Search papers, labs, and topics across Lattice.
This paper introduces Geo-EVS, a geometry-conditioned framework for extrapolative novel view synthesis in autonomous driving, designed to address the degradation of existing methods outside recorded trajectories. Geo-EVS employs Geometry-Aware Reprojection (GAR) using a fine-tuned VGGT to reconstruct and reproject colored point clouds, generating geometric condition maps. It further uses Artifact-Guided Latent Diffusion (AGLD) to inject reprojection-derived artifact masks during training, enabling the model to recover structure under missing support.
Synthesizing novel views from extrapolated poses no longer requires dense supervision, thanks to a geometry-conditioned diffusion model that explicitly learns to handle out-of-trajectory artifacts.
Extrapolative novel view synthesis can reduce camera-rig dependency in autonomous driving by generating standardized virtual views from heterogeneous sensors. Existing methods degrade outside recorded trajectories because extrapolated poses provide weak geometric support and no dense target-view supervision. The key is to explicitly expose the model to out-of-trajectory condition defects during training. We propose Geo-EVS, a geometry-conditioned framework under sparse supervision. Geo-EVS has two components. Geometry-Aware Reprojection (GAR) uses fine-tuned VGGT to reconstruct colored point clouds and reproject them to observed and virtual target poses, producing geometric condition maps. This design unifies the reprojection path between training and inference. Artifact-Guided Latent Diffusion (AGLD) injects reprojection-derived artifact masks during training so the model learns to recover structure under missing support. For evaluation, we use a LiDAR-Projected Sparse-Reference (LPSR) protocol when dense extrapolated-view ground truth is unavailable. On Waymo, Geo-EVS improves sparse-view synthesis quality and geometric accuracy, especially in high-angle and low-coverage settings. It also improves downstream 3D detection.
Tsinghua AI