Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ByDeWay-V2, an enhanced framework for improving spatial reasoning in Multimodal Large Language Models (MLLMs) by integrating explicit spatial relational context with depth cues, enabling better handling of object-to-object relationships. The approach addresses the limitations of prior methods that struggled with fine-grained spatial understanding and object hallucination, particularly in decision-critical applications. Evaluations on the Visual Spatial Reasoning and BLINK benchmarks demonstrate a significant 46% relative F1 improvement over previous methods, highlighting the framework's effectiveness in real-time, resource-constrained environments.
ByDeWay-V2 transforms spatial reasoning in MLLMs, achieving a 46% boost in accuracy while operating within strict resource constraints.
As Multimodal Large Language Models (MLLMs) are increasingly deployed in decision-critical pipelines such as robotics, embodied AI, and safety monitoring, the opacity of their spatial judgments limits operator trust and auditability. MLLMs demonstrate strong reasoning but often struggle with fine-grained spatial understanding and object hallucination. Prior work, ByDeWay, introduced Layered-Depth-Based Prompting (LDP), a training-free framework that mitigates hallucinations by structuring prompts using monocular depth estimation. However, coarse depth layering falls short in resolving object-to-object spatial relationships within the same geometric plane, such as projective ("left of", "above") and topological ("inside", "touching") relations. We propose ByDeWay-V2, which integrates explicit spatial relational context alongside depth cues, expressed as human-readable predicates that serve as auditable evidence for downstream decision support. Using an open-vocabulary object detector (YOLO-World-L), our framework computes pairwise geometric relations between detected objects and injects them as structured spatial predicates into the MLLM prompt, bridging 3D scene depth and 2D spatial semantics without any training. We evaluate ByDeWay-V2 on the Visual Spatial Reasoning (VSR) and BLINK benchmarks across multiple MLLMs, with hallucination grounding assessed via POPE. On the BLINK spatial subset, ByDeWay-V2 achieves a 46 percent relative F1 improvement over LDP for Qwen2.5-VL, and recovers BLIP-Base's spatial reasoning on VSR from near-random performance to a competitive F1 of 0.53. Our lightest configuration operates under a strict 40-token context budget on CPU, showing the framework's suitability for resource-constrained, real-time decision-support settings.