Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of spatial context prediction in vision-language models (VLMs) by introducing Spatial-TRM, a recursive refinement approach that leverages frozen vision-language features. By employing weight sharing and a convex region decoder, Spatial-TRM recursively reasons over spatial relationships and physical constraints to identify vacant regions. The model achieves a significant 44.0% relative improvement in context accuracy on the RoboSpatial-Home benchmark compared to a single-step baseline with identical trainable parameters.
Recursive refinement unlocks a 44% relative gain in spatial context prediction accuracy for VLMs, even with minimal trainable parameters.
Spatial context prediction requires visual grounding and spatial reasoning to identify vacant regions relative to objects in a scene. Spatial relationships and physical constraints remain challenging for large vision-language models (VLMs). Recursive reasoning has demonstrated strong performance on discrete and abstract reasoning tasks, motivating its use for improving spatial context prediction in natural images. Weight sharing enables higher accuracy with a small trainable parameter count. We introduce Spatial-TRM, a compact recursive model with frozen vision-language features, a convex region decoder, and just 11.2M trainable parameters. This weight-tied reasoner supports six spatial relationships, three reference frames, and open-vocabulary anchor objects. On the RoboSpatial-Home benchmark, Spatial-TRM achieves 29.51% context accuracy, a gain of 9.02 percentage points (44.0% relative) over a separately trained single-step model with the same parameter count. Including the frozen encoders, the model has 877.8M total parameters, approximately $15\times$ fewer than the reported 13B size of RoboPoint. These results show that repeated refinement improves spatial context prediction from pretrained vision-language features at a fixed trainable parameter count on this benchmark.