Search papers, labs, and topics across Lattice.
This paper conducts a comparative architectural analysis of multimodal question answering by benchmarking three foundational paradigms鈥擬ultimodal Adaptive Extraction (MAE), Solar, and UniMMQA鈥攁cross text, tabular, and visual modalities. Evaluating this progression is critical for determining whether multimodal reasoning should rely on specialized modality-specific pipelines or direct language-model-based unification. The authors find that unified, text-centric formulations (UniMMQA) systematically outperform multi-stage adaptive pipelines on Exact Match and F1 metrics, though linearizing non-text modalities introduces persistent information loss and weakens fine-grained cross-modal grounding.
Collapsing heterogeneous modalities like tables and images into unified text tokens outperforms specialized modular extractors across multimodal QA benchmarks, despite inevitable information loss during serialization.
The rapid growth of multimodal data has intensified the need for question answering (QA) systems capable of reasoning across heterogeneous sources such as text, tables, and images. In this paper, we present a comprehensive methodological comparison of three influential frameworks, namely Multimodal Adaptive Extraction (MAE), Solar, and UniMMQA, tracing the evolution of multimodal question answering from modality-adaptive pipelines to fully unified architectures. We examine how each approach models cross-modal interactions, transforms heterogeneous inputs, and performs reasoning, highlighting key design differences in modality representation, reasoning, and answer generation. Our analysis demonstrates a clear shift from explicit modality-specific processing toward unified text-centric formulations enabled by pre-trained language models (PLMs). Empirical comparisons across benchmark datasets show that this transition leads to substantial improvements in both Exact Match (EM) and F1-Scores, with UniMMQA achieving the most consistent and scalable performance. Despite these advances, we identify persistent challenges, including information loss during modality transformation, error propagation in multi-stage pipelines, and limitations in capturing fine-grained cross-modal dependencies. Overall, this study provides a deeper understanding of current design trends and offers insights into the future direction of unified multimodal reasoning systems.
