Search papers, labs, and topics across Lattice.
This paper introduces generative semantic scene completion (GSSC) for outdoor LiDAR semantic scene completion, addressing the challenge of recovering dense voxel grids from sparse observations with extreme class imbalance. The authors employ a discrete-diffusion framework that includes paired sparse-dense scene synthesis (PS^3) to create a new training dataset and semantic-guided generative scene completion (SGSC) to generate scenes conditioned on sparse scans. Notably, their structured source discrete diffusion (S^2D^2) method enhances the mean Intersection over Union (mIoU) of existing models, achieving a new state-of-the-art score of 38.8% on the SemanticKITTI leaderboard without retraining or test-time adaptation.
Achieving a new state-of-the-art mIoU of 38.8% in semantic scene completion with a single-sweep, single-sample approach reveals the power of generative diffusion methods in handling extreme class imbalance.
Outdoor LiDAR semantic scene completion (SSC) recovers a dense semantic voxel grid from a scan observing 1% of the target volume, under class imbalance beyond 7,000x. We recast SSC as generative semantic scene completion (GSSC): a single discrete-diffusion formulation in three roles. First, paired sparse-dense scene synthesis (PS^3) generates matched sparse LiDAR observations with their dense semantic completions, addressing the long tail at its source and yielding the PS^3-SemanticKITTI corpus we train on alongside SemanticKITTI. Second, semantic-guided generative scene completion (SGSC) generates the scene from noise with multinomial discrete diffusion, conditioned on the sparse scan through a bird's-eye-view semantic map and a sparse 3D feature stream. Third, the same framework instead refines an existing completion in one flow-matching step: structured source discrete diffusion (S^2D^2). S^2D^2 improves the mIoU of SGSC's own output and every external SSC base tested, without base retraining or test-time adaptation. On the strongest base, one step without test-time augmentation reaches 38.8% mIoU on the SemanticKITTI hidden test. To our knowledge that is the best causal, single-sweep, single-sample result on that leaderboard, +2.1 pp over the previous best published score under the same restriction. Four correction steps with eight-view test-time augmentation reach 39.2%, outside that restriction.
