Search papers, labs, and topics across Lattice.
The paper introduces GUIDE, a framework for progressively injecting geometric priors into the early layers of Multimodal Large Language Models (MLLMs) to improve spatial reasoning. GUIDE leverages multi-level sampling within a geometric encoder to capture features from local edges to global topologies, which are then aligned and fused with MLLM layers. Experiments show that GUIDE significantly outperforms existing methods on spatial reasoning and perception tasks by enabling the model to learn the 2D-to-3D transitional process and suppressing geometric noise.
MLLMs struggle with spatial reasoning because they flatten geometric information too early; GUIDE fixes this by progressively injecting multi-granularity geometric priors into early layers, leading to substantial performance gains.
Multimodal Large Language Models (MLLMs) have achieved remarkable progress in 2D visual tasks but still exhibit limited physical spatial awareness when processing real-world visual streams. Recently, feed-forward geometric foundation models, which implicitly extract geometric priors, have provided a new pathway to address this issue. However, existing geometry-aware MLLMs are predominantly constrained by the paradigm of single deep-layer extraction and input-level fusion. This flattened fusion leads to the loss of local geometric details and causes semantic mismatches in the early layers. To break this bottleneck, we propose GUIDE (Geometric Unrolling Inside MLLM Early-layers), a progressive geometric priors injection framework. GUIDE performs multi-level sampling within the geometric encoder, comprehensively capturing multi-granularity features ranging from local edges to global topologies. Subsequently, we rigorously align and fuse these multi-level geometric priors step-by-step with the early layers of the MLLM. Building upon the injection of multi-granularity geometric information, this design guides the model to progressively learn the 2D-to-3D transitional process. Furthermore, we introduce a context-aware gating that enables the model to fetch requisite spatial cues based on current semantics, thereby maximizing the utilization efficiency of spatial priors and effectively suppressing redundant geometric noise. Extensive experiments demonstrate that GUIDE significantly outperforms existing baselines on multiple complex spatial reasoning and perception tasks, establishing a novel paradigm for integrating 3D geometric priors into large models.
