Search papers, labs, and topics across Lattice.
To eliminate the entangled failure modes of joint semantic reasoning and spatial grounding in vision-language models, SeGDeP decouples the MLLM-segmenter interface into an explicit semantic feature branch and an independent DETR-predicted bounding-box path that jointly prompt a SAM 3 decoder. The architecture is trained through interface alignment followed by Group Reward-Decoupled Policy Optimization (GDPO), which isolates feedback across format, box-IoU, and mask-IoU rewards. Adapting just 0.38% of Qwen3-VL parameters with LoRA, SeGDeP-4B achieves 82.7 average cIoU across RefCOCO benchmarks and 66.0/59.6 gIoU on ReasonSeg val/test while demonstrating verifiable functional specialization across branches.
Reasoning segmentation models stumble by cramming semantic interpretation and spatial localization into a single prompt token; decoupling "what" from "where" hits state-of-the-art grounding while fine-tuning less than 0.4% of the base MLLM.
Reasoning segmentation converts an implicit linguistic conclusion into a precise mask, requiring both semantic identification and spatial grounding. Existing MLLM-segmenter interfaces either use a special trigger or compress both signals into one context, although they receive different supervision and fail differently. This coupling obscures whether a failure arises from target interpretation or from localization. We present SeGDeP, an explicit what-where interface. A semantic prompt branch and an independent geometric projection path transform resolved MLLM states into semantic features and a DETR-predicted box, which jointly condition a SAM 3 mask decoder. Training first aligns this executable interface, then uses group reward-decoupled policy optimization (GDPO) to balance format, box-IoU, and mask-IoU feedback. SeGDeP-4B reaches 82.7 average cIoU over eight RefCOCO-family splits and 66.0/59.6 gIoU on ReasonSeg val/test while adapting only 0.38% of Qwen3-VL parameters through LoRA. Controlled stage-wise ablations, gradient diagnostics, and prompt interventions further show that the two paths develop complementary semantic and geometric specialization rather than duplicating the same evidence.
