Search papers, labs, and topics across Lattice.
LiteMVS introduces a lightweight multi-view depth estimation model that combines plane-sweep geometric reasoning with high-level monocular semantic and structural priors to enhance 3D perception. By integrating a Mixture-of-Experts (MoE) framework for adaptive geometric aggregation and leveraging distilled geometric priors from vision foundation models, it significantly improves depth estimation and 3D reconstruction quality in static scenes. Experiments on ScanNetv2 and 7-Scenes show that LiteMVS achieves high-quality results while maintaining competitive efficiency, making it suitable for real-time applications in robotics and augmented reality.
LiteMVS achieves superior depth estimation and 3D reconstruction by merging monocular insights with multi-view geometric reasoning, all while keeping inference costs low.
Real-time 3D perception is crucial for robotics, augmented reality, and embodied intelligence applications. Existing multi-view stereo (MVS) methods primarily rely on geometric correspondences, which often fail in textureless or repetitive regions, while monocular depth models leverage strong image-level priors but lack robust multi-view geometric constraints. More importantly, in robotics and embodied manipulation scenarios, high-quality 3D geometry is not only essential for static reconstruction, but also serves as a critical foundation for learning temporally consistent 4D representations. To obtain visual representations with stronger structural awareness and greater potential for spatiotemporal extension, we present LiteMVS, a lightweight multi-view depth estimation model that integrates plane-sweep geometric reasoning with strong monocular semantic and structural priors. The central idea of LiteMVS is to efficiently inject high-level monocular knowledge, obtained from lightweight segmentation models and large-scale vision foundation models, into a multi-view stereo framework. In particular, LiteMVS enriches the cost volume with semantic descriptors and employs a Mixture-of-Experts (MoE) formulation to enable adaptive geometric aggregation across depth hypotheses. Moreover, geometric priors distilled from vision foundation models further strengthen monocular guidance without increasing inference cost. Through this design, LiteMVS not only improves depth estimation and 3D reconstruction quality in static scenes, but also provides a more reliable geometric foundation for subsequent temporal modeling and 4D representation learning. Experiments on ScanNetv2 and 7-Scenes demonstrate that LiteMVS achieves high-quality depth prediction and 3D reconstruction while maintaining competitive efficiency.
