Search papers, labs, and topics across Lattice.
This paper introduces Cycle Group Relative Policy Optimization (CycleGRPO), a novel reinforcement learning framework that simultaneously enhances region understanding and localization in Multimodal Large Language Models (MLLMs) by allowing the model to act as both actor and critic. By leveraging a self-evaluating paradigm that transitions from generating region captions to grounding them in spatial contexts, CycleGRPO eliminates the need for textual ground truths, relying solely on region inputs. The empirical results demonstrate significant performance improvements across various tasks, including region captioning and grounded dialogue, without requiring task-specific fine-tuning.
CycleGRPO achieves simultaneous region understanding and localization in MLLMs without any reliance on textual ground truths, revolutionizing multimodal task integration.
This paper introduces Actor as Its Own Critic, a unified reinforcement learning framework, Cycle Group Relative Policy Optimization (CycleGRPO), that jointly optimizes region understanding and localization for Multimodal Large Language Models (MLLMs). Unlike existing separate pipelines, we leverage the inherent duality between the two tasks to construct a self-evaluating reinforcement learning paradigm: "region $\to$ text $\to$ region''. Specifically, a single MLLM first acts as the actor to generate region captions, then immediately transitions to a critic to ground its generated text back in the spatial domain. Therefore, CycleGRPO requires only region inputs, e.g., masks or bounding boxes, entirely bypassing the need for textual ground truths. A quality-aware token-level cycle-consistency reward is employed to assess the semantic discriminability of text captions via their physical localization accuracy. Empirically, built upon SAMTok, our CycleGRPO framework successfully bootstraps both capabilities simultaneously. Without any task-specific fine-tuning, the framework yields consistent performance gains across a wide range of benchmarks, including region captioning, region VQA, grounded dialogue, and referring segmentation. Overall, CycleGRPO offers a straightforward and scalable way to advance pixel-level capabilities in MLLMs. Code and models are released at https://github.com/devinxzhang/CycleGRPO.
NUS