Search papers, labs, and topics across Lattice.
To mitigate the quadratic token explosion of multi-view 3D scene reasoning in 2D VLMs, the authors develop CoVeR, a deterministic, training-free token pruning framework driven strictly by 3D spatial coordinates rather than learned feature importance. This geometric formulation resolves the spatial clustering traps of attention-based selection while bypassing the rigid budget saturation plateaus seen in traditional voxelization methods. Evaluated across four VLMs and three benchmarks, CoVeR preserves 93.5% of full-token performance while pruning roughly 92% of visual tokens, outperforming existing multi-view token pruning SOTAs by an average of 3.9 percentage points.
Throwing away learned attention in favor of pure 3D spatial coverage allows VLMs to retain over 93% of their multi-view 3D reasoning performance while slashing visual token budgets by ~92%.
Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotated 3D data. However, it produces thousands of redundant visual tokens whose cost grows with every view. Existing visual token pruners fall into two families, each limited in the 3D multi-view setting. Learned importance methods rank tokens by attention or encoder features; because redundancy here is fundamentally spatial, they keep near-duplicate tokens from a few prominent regions and leave most of the scene unrepresented. Voxelization methods improve spatial coverage but cannot enforce an exact token budget and saturate as multi-view observations overlap in 3D, capping retention well below the target. We show that spatial coverage is associated with 3D reasoning performance and introduce CoVeR, a deterministic, training-free selector that uses only token coordinates, with no learned signals. CoVeR selects tokens that collectively cover every region of the scene, and solves the limitations of both families: it enforces an exact per-scene budget, breaks the voxelization saturation plateau, and avoids the near-duplicate selections of learned importance. Extensive experiments show CoVeR outperforms prior SOTAs on all three 3D reasoning benchmarks and generalizes as a plug-and-play module tested across four VLMs. Notably, with only $\approx$8% of visual tokens, it preserves 93.5% of full-token performance, surpassing SOTA by 3.9 percentage points on average across benchmarks.
