Search papers, labs, and topics across Lattice.
This paper introduces Hyper-M2RAG, a framework that leverages High-order Hypergraph Representation Learning to enhance Multimodal Retrieval-Augmented Generation (M-RAG) systems by capturing complex N-ary relationships among heterogeneous entities. By employing an Anchor-driven Incremental Refinement mechanism, the method addresses the inefficiencies of traditional full-page reconstruction approaches, allowing for targeted refinement of local hyper-topologies while minimizing computational costs. Extensive evaluations reveal that Hyper-M2RAG significantly improves retrieval precision and generation coherence compared to existing state-of-the-art techniques.
Hyper-M2RAG redefines multimodal retrieval by capturing complex relationships in a hypergraph structure, achieving superior performance with less computational overhead.
Modern Multimodal Retrieval-Augmented Generation (M-RAG) systems are fundamentally limited by the binary connectivity paradigm of traditional simple graphs, which fails to capture the intricate, high-order correlations among heterogeneous entities, such as the N-ary relationships between a visual chart, its scattered textual descriptions, and underlying numerical data. Furthermore, existing refinement strategies often rely on exhaustive, full-page reconstruction to align cross-modal information, leading to prohibitive computational redundancy and the introduction of contextual noise in long-form document processing. In this paper, we propose Hyper-M2RAG, a novel framework that redefines multimodal document retrieval through High-order Hypergraph Representation Learning. We first formalize the document structure as a Multimodal Hypergraph, utilizing hyperedges as unified semantic containers to encapsulate multi-way associations across text, images, and tables, thereby transcending point-to-point modeling. To mitigate semantic fragmentation caused by physical pagination, we introduce an Anchor-driven Incremental Refinement mechanism. Rather than performing a global sweep, our approach identifies boundary-crossing anchor nodes and reconstructs their local hyper-topology using one-hop neighborhood contexts. This targeted refinement effectively bridges cross-page knowledge gaps with minimal computational footprints. Extensive evaluations on multimodal benchmarking datasets demonstrate that Hyper-M2RAG significantly outperforms state-of-the-art methods in both retrieval precision and generation coherence. Our code is available at https://github.com/ShenAoChen2001/MMHRAG.
