Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to synthesizing coherent 3D Gaussian street scenes from imperfect 2D anchors by employing teacher-relative appearance residual distillation for appearance baking. The method effectively addresses the challenges of view-specific noise and the limitations of existing pipelines by utilizing a structured space for frequency decomposition and confidence estimation, leading to improved target alignment and content preservation. Evaluation results demonstrate that the proposed technique outperforms editing-based baselines in terms of artifact suppression and cross-view consistency across various datasets, including Waymo street assets and Tanks and Temples scenes.
Achieving coherent 3D scene synthesis from imperfect 2D anchors could redefine the standards for visual fidelity in urban modeling.
Image priors can synthesize target conditions for 3D Gaussian street scenes, but independently edited views do not define a coherent 3D target. Direct fitting can propagate view-specific noise, while existing pipelines do not jointly handle imperfect sparse anchors and standard-rasterizer deployment. To address this gap, teacher-relative appearance residual distillation is introduced for appearance baking. A structured space for frequency decomposition, confidence estimation, and primitive-level lifting is formed by residuals between teacher anchors and original renders. The direct optimization signal is supplied by renderer-space matching, while primitive assignment is regularized by support-aware Gaussian-space aggregation. Supported detail is admitted and unsupported noise is suppressed through confidence-gated coarse-to-fine optimization, after which all residuals are baked into fixed-geometry spherical-harmonic coefficients. The teacher and auxiliary training modules are discarded at inference. Evaluation across Waymo street assets, Tanks and Temples scenes, and multiple target conditions shows a favorable overall balance of target alignment, content preservation, artifact suppression, and cross-view consistency over editing-based baselines. Ablations confirm the effectiveness of the main components. Code will be released at https://github.com/Cagares/Baking-for-3D-Gaussian.
