Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ObjectSplat, a novel approach to reconstructing 3D scenes that enhances mesh fidelity and interactivity by employing a decompose-before-reconstruct strategy. By segmenting instances from input frames and treating the background separately, the method effectively inpaints unobserved regions and reconstructs both instances and background with improved mesh splatting techniques. The results show over a 5% gain in F-score for mesh fidelity and enable object-level modifications, significantly advancing the usability of 3D reconstructions for interactive applications.
ObjectSplat achieves over a 5% improvement in mesh fidelity while enabling real-time object-level editing in 3D scene reconstructions.
Splatting-based algorithms reconstruct photorealistic, real-time-renderable, and mesh-exportable 3D scenes from regular images, but they represent a scene as a single monolithic field. Therefore, the reconstruction has no object-level structure, leaving it infeasible for downstream editing or interaction. Moreover, regions that are never directly observed in the input scans are contaminated by the surrounding texture and left uncorrected, capping both mesh fidelity and novel-view synthesis. We propose a decompose-before-reconstruct approach: we segment the instances out of every frame, consider the remaining as background and inpaint it, reconstruct each instance and the background independently with mesh splatting, and compose them into a single scene. Our method significantly improves mesh fidelity (over a 5\% gain in F-score) and novel-view synthesis, while supporting object-wise modifiability and interactivity. The code will be made publicly available.