Search papers, labs, and topics across Lattice.
This paper introduces AnchorSeg, a novel approach to reasoning segmentation that utilizes structured conditional generation over image tokens, leveraging language-grounded query banks to separate semantic reasoning from spatial localization. By employing an ordered sequence of latent reasoning tokens and a segmentation anchor token, the method enhances the model's ability to explicitly determine what to segment and where to segment. The proposed Token鈥揗ask Cycle Consistency (TMCC) training objective aligns predictions across resolutions, leading to state-of-the-art performance on the ReasonSeg test set with 67.7% gIoU and 68.1% cIoU.
By decoupling semantic reasoning from spatial grounding, AnchorSeg achieves unprecedented accuracy in reasoning segmentation tasks.
Reasoning segmentation requires models to ground complex, implicit textual queries into precise pixel-level masks. Existing approaches rely on a single segmentation token $\texttt{<SEG>}$, whose hidden state implicitly encodes both semantic reasoning and spatial localization, limiting the model's ability to explicitly disentangle what to segment from where to segment. We introduce AnchorSeg, which reformulates reasoning segmentation as a structured conditional generation process over image tokens, conditioned on language grounded query banks. Instead of compressing all semantic reasoning and spatial localization into a single embedding, AnchorSeg constructs an ordered sequence of query banks: latent reasoning tokens that capture intermediate semantic states, and a segmentation anchor token that provides explicit spatial grounding. We model spatial conditioning as a factorized distribution over image tokens, where the anchor query determines localization signals while contextual queries provide semantic modulation. To bridge token-level predictions and pixel-level supervision, we propose Token--Mask Cycle Consistency (TMCC), a bidirectional training objective that enforces alignment across resolutions. By explicitly decoupling spatial grounding from semantic reasoning through structured language grounded query banks, AnchorSeg achieves state-of-the-art results on ReasonSeg test set (67.7\% gIoU and 68.1\% cIoU). All code and models are publicly available at https://github.com/rui-qian/AnchorSeg.
