Search papers, labs, and topics across Lattice.
This paper introduces shell-supervised Gaussian Splatting, a novel framework for video-driven 3D reconstruction that leverages an external facade structural shell to enhance geometric accuracy in urban environments. By aligning the shell to video frames and utilizing mask-gated losses, the method effectively improves the consistency of facade orientation and visible-surface point clouds, addressing challenges posed by reflections and weak textures. Experimental results demonstrate that this approach outperforms traditional methods while maintaining high rendering quality, making it a significant advancement in real-to-sim reconstruction for embodied AI applications.
Urban facade reconstruction can achieve superior geometric accuracy by integrating lightweight structural supervision, overcoming common pitfalls of traditional methods.
Real-to-sim reconstruction for embodied AI requires geometry that is useful for collision reasoning, navigation, and agent-environment interaction, not only photorealistic novel-view synthesis. However, close-range urban facades are difficult for video-to-3D reconstruction: glass, reflections, repeated windows, and weak texture can produce visually plausible renderings with unstable surface geometry. We introduce shell-supervised Gaussian Splatting, a reconstruction-stage framework that uses an external facade structural shell as lightweight geometric supervision for video-driven Gaussian reconstruction. The method aligns an exterior shell to the video reconstruction frame, renders per-view depth, camera-space normal, and valid-mask maps, and applies these cues through mask-gated losses during Gaussian optimization. This design preserves RGB-driven appearance while regularizing only visible shell-supported facade regions. Experiments on anonymized close-range urban facade scenes show improved facade orientation and visible-surface point-cloud consistency over photo-only, monocular-cue, and surface-oriented Gaussian baselines, while maintaining comparable held-out rendering quality.
Tsinghua AI