Search papers, labs, and topics across Lattice.
This paper introduces Learning Gaussian Structure (LGS), a novel framework that improves Gaussian reconstruction for driving scenes by dynamically adjusting Gaussian primitives based on intervention-guided density control. By leveraging local gradient responses from pruning and adding interventions, LGS learns a Densify Map that optimally modifies the Gaussian structure during inference, enhancing both the representation and attribute prediction. Experimental results on the Waymo Open Dataset and PandaSet show that LGS significantly outperforms traditional methods, marking a substantial advancement in scene reconstruction techniques.
Intervention-guided density control allows for real-time optimization of Gaussian structures, leading to superior scene reconstruction performance.
Feed-forward Gaussian reconstruction has recently emerged as an efficient approach for driving scene reconstruction. However, prevailing LiDAR-based methods preserve the initial correspondence between observed points and Gaussian primitives, treating the initialized primitive set as the final representation. Unlike optimization-based 3DGS, these methods cannot accumulate gradients during training to determine how the scenes representation should be densified. Meanwhile, the shared sparse backbone only fuses observations from different timestamps implicitly, without explicitly aggregating cross-time evidence for individual primitives. In this paper, we present Learning Gaussian Structure (LGS), a framework that enhances both Gaussian structure and primitive attributes. Our key observation is that changes in local gradient responses induced by a prune or add intervention reveal whether the corresponding structural adjustment benefits reconstruction. Based on this observation, our Gaussian Densify Policy learns a Densify Map comprising Prune and Addition Scores from controlled interventions, and directly adjusts the Gaussian structure during inference. We further develop a compact Cross-Time Point Query that explicitly retrieves and aggregates neighboring features from Gaussian primitives at other timestamps for reliable attribute prediction. Extensive experiments on the Waymo Open Dataset and PandaSet demonstrate that LGS consistently outperforms existing methods.
