Search papers, labs, and topics across Lattice.
To overcome the dimensional mismatch between 2D visual projections and 3D physical reality, FactoSR factorizes complex 4D spatial reasoning into three verifiable reinforcement learning sub-objectives: planar correspondence ($XY$), depth consistency ($Z$), and temporal reversibility ($T$). This divide-and-conquer strategy transforms the ill-posed challenge of recovering collapsed geometric dimensions into a structured, step-by-step reasoning policy. The approach yields consistent gains across multi-view and video benchmarks, improving performance by 5.9% on VSI-Bench and 4.5% on All-Angles-Bench.
Overcoming the 2D-to-4D spatial bottleneck in VLMs does not require native 3D architectures; factorizing visual projections into verifiable planar, depth, and temporal RL objectives delivers immediate 4-6% benchmark gains.
Despite the remarkable prowess of Vision-Language Models (VLMs) in general multimodal tasks, they remain fundamentally ``flat''when reasoning about the physical world. We argue that this spatial bottleneck stems from a profound dimensional mismatch: while VLMs are trained to interpret 2D projections, true spatial reasoning demands the recovery of latent 3D geometry and temporal continuity. To conquer this high-dimensional complexity, we advocate a shift from monolithic learning to a ``divide and conquer''paradigm. We present FactoSR, a factorized reinforcement learning framework that explicitly interpret the dimensions collapsed by visual projection. At its core, FactoSR decomposes the monolithic problem of world-consistent reasoning into three orthogonal, geometric sub-objectives: planar correspondence ($XY$), depth consistency ($Z$), and temporal reversibility ($T$). By optimizing these verifiable constraints within a unified policy learning mechanism, we effectively transform an ill-posed projection recovery problem into a series of tangible reasoning steps. Extensive evaluations on multi-view and video benchmarks demonstrate that this elegant decomposition yields substantial gains in 3D and 4D reasoning, achieving a 5.9% boost on VSI-Bench and 4.5% on All-Angles-Bench. Our findings suggest that reinforcing explicit, factorized 4D consistency is a critical step toward evolving VLMs into robust, world-aware reasoners.
