Search papers, labs, and topics across Lattice.
The paper introduces Latent Gaussian Splatting (LaGS) for 4D Panoptic Occupancy Tracking, addressing the need for detailed spatiotemporal scene understanding for robotics. LaGS fuses multi-view camera observations into 3D Gaussians, which are then splatted onto a 3D voxel grid and decoded by a mask-based segmentation head for panoptic occupancy prediction. Experiments on Occ3D nuScenes and Waymo datasets demonstrate state-of-the-art performance in 4D panoptic occupancy tracking.
Achieve state-of-the-art 4D scene understanding by representing dynamic environments as a collection of Gaussians that are efficiently splatted onto a voxel grid for panoptic occupancy tracking.
Capturing 4D spatiotemporal surroundings is crucial for the safe and reliable operation of robots in dynamic environments. However, most existing methods address only one side of the problem: they either provide coarse geometric tracking via bounding boxes, or detailed 3D structures like voxel-based occupancy that lack explicit temporal association. In this work, we present Latent Gaussian Splatting for 4D Panoptic Occupancy Tracking (LaGS) that advances spatiotemporal scene understanding in a holistic direction. Our approach incorporates camera-based end-to-end tracking with mask-based multi-view panoptic occupancy prediction, and addresses the key challenge of efficiently aggregating multi-view information into 3D voxel grids via a novel latent Gaussian splatting approach. Specifically, we first fuse observations into 3D Gaussians that serve as a sparse point-centric latent representation of the 3D scene, and then splat the aggregated features onto a 3D voxel grid that is decoded by a mask-based segmentation head. We evaluate LaGS on the Occ3D nuScenes and Waymo datasets, achieving state-of-the-art performance for 4D panoptic occupancy tracking. We make our code available at https://lags.cs.uni-freiburg.de/.
