Search papers, labs, and topics across Lattice.
This paper introduces a novel structure-aware merging pipeline that consolidates per-pixel primitives from feed-forward 3D Gaussian methods into a compact, content-adaptive Gaussian set, achieving a reduction to just 1/20th of the original Gaussian count. By utilizing adaptive superpixel segmentation guided by saliency maps, the method groups spatially coherent Gaussians based on appearance, allowing for efficient representation without significant loss in visual quality. The approach is backbone-agnostic and enhances rendering efficiency while improving robustness compared to previous primitive reduction techniques.
Achieving a staggering 95% reduction in Gaussian count without sacrificing visual fidelity, this method transforms 3D scene reconstruction efficiency.
3D scene reconstruction, modeling, and rendering are highly relevant for numerous tasks, and 3D Gaussian splatting has become a standard choice in this context. Its feed-forward variants provide fast reconstruction from sparse input views but often produce per-pixel primitives, leading to highly redundant and thus inefficient representations. We present a structure-aware merging pipeline that takes per-pixel primitives from any feed-forward method and consolidates them into a compact, content-adaptive Gaussian set while largely retaining visual quality at just $\frac{1}{20}^\text{th}$ of the Gaussians of a per-pixel method. We group spatially coherent Gaussians of similar appearance into variable-size clusters via adaptive superpixel segmentation guided by a saliency map, which allocates fine segments to textured regions and coarse segments to homogeneous areas. We compress each cluster into a compact latent representation through a learned encoder, then match and consolidate representations across views based on geometric overlap and feature similarity via a learned merger. A level-of-detail decoder then produces the final Gaussians at a controllable resolution, enabling a flexible quality-efficiency trade-off at inference. As a post-processing module, the pipeline is backbone-agnostic, leveraging the strengths of existing feed-forward methods. This leads to better and more robust quality than achieved by previous approaches that target a reduction in primitive count, while providing a highly compact representation, that can be rendered efficiently.
