Search papers, labs, and topics across Lattice.
This paper introduces Symbal, a dual-stage detection system designed to identify systematic misalignments in captions generated by multimodal large language models (MLLMs), where specific visual features lead to recurring captioning errors. By leveraging a large vision-language dataset, Symbal achieves a remarkable 63.8% accuracy in detecting these misalignments, significantly outperforming existing baselines by nearly four times. Additionally, the authors present SymbalBench, a comprehensive benchmark comprising 1.7 million image-text pairs that facilitates the evaluation of automated methods for this task.
Systematic misalignments in MLLM-generated captions can be detected with 63.8% accuracy, revealing a critical flaw in image-text pairing that has been largely overlooked.
Multimodal large language models (MLLMs) often introduce errors when generating image captions, resulting in misaligned image-text pairs. Our work focuses on a class of captioning errors that we refer to as systematic misalignments, where a recurring error in MLLM-generated captions is closely associated with the presence of a specific visual feature in the paired image. Given a vision-language dataset with MLLM-generated captions, our aim in this work is to detect such errors, a task we refer to as systematic misalignment detection. As our first key contribution, we present Symbal, which utilizes a structured, dual-stage setup with off-the-shelf foundation models to identify systematic misalignments and summarize results in natural language. As our second key contribution, we introduce SymbalBench, a benchmark designed to evaluate automated methods on our proposed task. SymbalBench consists of 1.7 million image-text pairs from two domains (natural and medical images), organized into 420 vision-language datasets with annotated systematic misalignments. Symbal exhibits strong performance on this benchmark, correctly identifying systematic misalignments in 63.8% of datasets, a nearly 4x improvement over the closest baseline. We supplement our evaluations on SymbalBench with real-world evaluations, showing that (1) Symbal can accurately surface systematic misalignments in captions generated by four MLLMs and (2) Symbal is a powerful tool for auditing off-the-shelf image-caption datasets. Ultimately, our novel task, method, and benchmark can aid users with auditing MLLM-generated captions and identifying critical errors, without requiring access to the underlying MLLM. Code is available at https://github.com/Stanford-AIMI/Symbal.
Stanford HAI