Search papers, labs, and topics across Lattice.
CogniVerse, a new multi-modal retrieval-augmented generation (MMRAG) framework, tackles limitations in existing systems by integrating cognitive reflection for dynamic retrieval assessment, Riemannian manifold alignment for precise retrieval, and optimal transport-based loss for coherent generation. The Cognitive Reflection Module filters irrelevant multi-modal content, the Multi-modal Retrieval Module aligns embeddings using information geometry and refines knowledge graphs via spectral graph theory, and the Hierarchical Generation Module balances token-level accuracy and global semantic coherence. Experiments show CogniVerse outperforms state-of-the-art systems in accuracy and coherence while reducing retrieval latency.
MMRAG gets a human-like reasoning upgrade: CogniVerse uses cognitive reflection and information geometry to filter noise, align modalities, and generate coherent responses, outperforming existing systems.
Multi-modal Retrieval-Augmented Generation (MMRAG) has emerged as a powerful paradigm for enhancing Multimodal Large Language Models in knowledge-intensive question answering by integrating external visual, textual, and structural knowledge. However, existing MMRAG frameworks suffer from critical limitations, including noisy and irrelevant retrieval, cross-modal semantic misalignment, lack of adaptive reasoning, and incoherent generation across local and global contexts. We introduce \textbf{CogniVerse}, a novel MMRAG framework that addresses these challenges through a cognitive-inspired, mathematically rigorous approach. Drawing from human-like reasoning, CogniVerse integrates three synergistic components: (1) a Cognitive Reflection Module that dynamically assesses retrieval necessity and filters relevant multi-modal content, reducing noise and computational overhead; (2) a Multi-modal Retrieval Module that aligns embeddings in a Riemannian manifold using information geometry and refines knowledge graphs via spectral graph theory, ensuring precise and coherent retrieval; and (3) a Hierarchical Generation Module that employs an optimal transport-based loss to balance token-level accuracy and global semantic coherence. Extensive experiments demonstrate that CogniVerse significantly outperforms state-of-the-art systems in both accuracy and coherence, while reducing retrieval latency.
