Search papers, labs, and topics across Lattice.
This paper introduces DGSeg, a novel reasoning segmentation framework that enhances pixel-wise mask predictions by dynamically fusing semantic and spatial cues generated by a Multimodal Large Language Model (MLLM). By employing separate segmentation branches for these cues and integrating them through a dynamic gating module, DGSeg effectively mitigates the impact of ambiguous or erroneous cues on the final predictions. Experimental results show that DGSeg outperforms existing methods, achieving state-of-the-art gIoU scores of 69.6% and 67.3% on the ReasonSeg validation and test splits, respectively.
Dynamic gating allows DGSeg to filter out noise and ambiguity in segmentation cues, leading to superior performance in reasoning segmentation tasks.
Reasoning segmentation aims to predict pixel-wise masks for targets given complex language queries. Existing approaches leverage Multimodal Large Language Models (MLLMs) for vision-language reasoning and generate intermediate target cues (e.g., points or boxes) to guide a segmentation model. However, compressing rich reasoning into sparse cues often introduces ambiguity and noise, preventing these cues from accurately preserving the reasoning intent. While multiple complementary cues can enrich target information, existing methods typically feed them jointly into a single segmentation process, allowing ambiguous or erroneous cues to affect the entire prediction. Therefore, we propose DGSeg, a reasoning segmentation framework that learns to fuse predictions guided by semantic and spatial cues. Specifically, the MLLM jointly reasons about both target identity and spatial location, producing complementary semantic and spatial cues that are fed into separate segmentation branches. Their predictions are adaptively integrated by a lightweight dynamic gating module trained with relative branch-quality supervision to suppress noisy or conflicting regions. Extensive experiments demonstrate that DGSeg consistently outperforms strong baselines on multiple benchmarks and achieves 69.6% and 67.3% gIoU on the challenging ReasonSeg validation and test splits. Code is available at https://github.com/RZZeng/DGSeg.
