Search papers, labs, and topics across Lattice.
The paper introduces DRBench, a new benchmark designed to evaluate dense-scene reasoning in VLMs, highlighting the failure of lightweight models in this area. To address this, they propose DRScaffold, a supervised fine-tuning framework that decomposes the reasoning process into causally ordered stages to enforce grounded reasoning. Experiments show that DRScaffold significantly improves performance on DRBench for lightweight VLMs, even surpassing the performance of a much larger frozen model.
Structured supervision can enable 3B VLMs to outperform 32B VLMs on dense-scene reasoning tasks, suggesting a path to efficient and reliable visual understanding.
Lightweight vision-language models perform competitively on standard benchmarks yet fail systematically in dense-scene reasoning, where multiple objects, attributes, and relations must be jointly grounded and resolved through multi-step inference. Such capability is critical for real-world applications where models must reliably interpret cluttered environments. Yet existing training signals provide no explicit grounding between reasoning steps and the underlying visual entities and relations, leaving lightweight models free to generate fluent but visually unanchored reasoning chains. To address this gap, we first introduce DRBench, a benchmark of 14,573 questions across 2,943 images, organized into five task categories spanning three progressive reasoning layers. Building on DRBench, we propose DRScaffold, a supervised fine-tuning framework that decomposes the supervision target into four causally ordered stages, enforcing grounded reasoning without architectural modification. Experiments on three lightweight VLMs demonstrate substantial gains on DRBench while preserving or improving performance on general-purpose benchmarks. Notably, Qwen2.5-VL-3B trained with DRScaffold surpasses the frozen Qwen2.5-VL-32B on DRBench, demonstrating that structured supervision can substitute for a significant portion of model scale in dense-scene reasoning. Our code and models are available at https://github.com/irene-shi/DRScaffold .
