Search papers, labs, and topics across Lattice.
This paper introduces Grounded Context Preference Optimization (Groc-PO), a novel framework designed to enhance the truthfulness of Multimodal Large Language Models (MLLMs) by addressing the propagation of reasoning errors across multiple grounding stages. By constructing the Grounded Context Preference Dataset (GCPD), the authors provide a structured approach to preference optimization that focuses on Object Grounding, Contextual Grounding, and Grounded Reasoning, thereby allowing for more explicit supervision at each stage. Experimental results demonstrate that Groc-PO significantly outperforms standard Direct Preference Optimization (DPO) and other baselines in reducing visual hallucinations and improving overall reliability in reasoning tasks.
Explicitly optimizing preferences at multiple grounding stages can dramatically reduce hallucinations and enhance the reliability of multimodal reasoning in LLMs.
Despite the rapid progress of Multimodal Large Language Models (MLLMs), they still suffer from untruthfulness issues, such as visual hallucinations, content fabrication, and unfaithful reasoning, which substantially undermine their faithfulness and practical utility. Alignment methods based on human preference, such as Direct Preference Optimization (DPO), have been widely adopted to address these issues. However, multimodal reasoning errors often propagate across stages, and final-answer errors can often be traced to mistakes in early grounding stages, yet standard DPO typically applies preference optimization at the final-answer level. This credit-assignment challenge means that supervision for early grounding stages is indirect rather than stage-specific, making it difficult to suppress error propagation arising from grounding drift and context inconsistency. To address this, we propose Grounded Context Preference Optimization (Groc-PO), a grounded preference optimization framework for MLLMs. We further construct the Grounded Context Preference Dataset (GCPD), organizing multi-stage preference samples around three stages of Object Grounding, Contextual Grounding, and Grounded Reasoning, to capture the formation, integration, and utilization of grounded context. By introducing more explicit preference supervision over multiple grounded stages, Groc-PO strengthens context-dependent reasoning and mitigates cross-stage error propagation. Extensive experiments show that, compared with standard DPO and other strong baselines, Groc-PO achieves improved performance in hallucination mitigation, faithful reasoning, and overall reliability, supporting the value of more explicit grounded supervision for trustworthy multimodal reasoning.
