Search papers, labs, and topics across Lattice.
The paper introduces Deformable Gaussian Occupancy (DeGO), a novel framework for weakly supervised dynamic 3D scene understanding that decouples rigid and nonrigid motion. DeGO represents scenes using Gaussian primitives that evolve through both deformation and offset-based updates, enabling the capture of fine-grained deformations. A factorized 4D distillation strategy transfers knowledge from the VGGT foundation model to enhance temporal consistency. DeGO achieves state-of-the-art performance on the Occ3D-NuScenes benchmark, demonstrating significant improvements in human-centric occupancy prediction.
Modeling dynamic 3D scenes gets a serious upgrade: DeGO's deformable Gaussians and foundation model distillation boost occupancy prediction accuracy by 13.5% on human-centric instances.
Understanding dynamic 3D environments is essential for safe autonomous driving, particularly when reasoning about human-centric, nonrigid agents. However, existing weakly supervised occupancy prediction frameworks predominantly assume rigid-body motion and rely on simple frame-to-frame offsets, limiting their ability to capture fine-grained deformations and maintain temporal coherence. To address this issue, we propose DeGO, a deformable Gaussian occupancy framework that unifies decoupled Gaussian deformation with factorized 4D foundation-model distillation. DeGO disentangles rigid and nonrigid motion, enabling each Gaussian primitive to evolve through both deformation and offset-based updates. In parallel, a factorized 4D distillation strategy transfers cross-camera and cross-frame knowledge from the VGGT foundation model, producing foundation-aligned features that enhance temporal consistency. Experiments on the Occ3D-NuScenes benchmark demonstrate that our method achieves state-of-the-art performance under weak supervision, delivering 13.5% gains on human-centric instances and 10.9% overall improvements. These results highlight the effectiveness of deformation-aware and foundation-guided occupancy modeling for dynamic scene understanding. The code is publicly available: https://github.com/vita-epfl/DeGO
