Search papers, labs, and topics across Lattice.
This paper introduces a training-free inference framework for VGGT that organizes views into diversity-aware balanced chunks using combinatorial graph partitioning based on visual dissimilarity and spatial dispersion. By focusing attention on geometrically informative views and minimizing redundant interactions, the method addresses the challenges of scaling VGGT to large view collections while maintaining reconstruction quality. Extensive experiments show significant improvements in camera pose estimation, multi-view depth prediction, and 3D reconstruction, alongside reductions in memory usage and inference latency.
Organizing views into diversity-aware chunks can drastically enhance the performance of geometry transformers while slashing memory costs and inference times.
Geometry transformers such as VGGT achieve strong performance by jointly reasoning over multiple views with global attention. However, scaling them to large view collections remains challenging due to the quadratic cost of attention. Moreover, our empirical analysis reveals that the reconstruction quality in VGGT is sensitive to the distribution of viewpoints. Simply increasing the number of views without sufficient viewpoint diversity can even degrade performance, as redundant views introduce highly similar tokens that dilute informative geometric signals in the attention mechanism. Motivated by this observation, we propose a training-free and plug-and-play VGGT inference framework that organizes views into diversity-aware balanced chunks. The chunks are constructed through combinatorial graph partitioning over visual dissimilarity and spatial dispersion. This view organization allows the transformer to focus attention on geometrically informative views while reducing redundant attention interactions. To estimate spatial dispersion without full pose estimation, we approximate spatial relationships via a soft pose propagation strategy based on visual similarity from a small set of seed frames. Extensive experiments demonstrate improved performance in camera pose estimation, multi-view depth prediction, and 3D reconstruction while reducing memory usage and inference latency. Our framework also complements existing VGGT variants, enabling scalable multi-view reconstruction without sacrificing geometric fidelity.
