Search papers, labs, and topics across Lattice.
This paper introduces M3Prune, a hierarchical communication graph pruning framework designed to enhance the efficiency of multi-modal retrieval-augmented generation (mRAG) systems by reducing token overhead and computational costs. By employing intra-modal and inter-modal graph sparsification techniques, M3Prune identifies and retains only the most critical communication links among multiple agents, thereby optimizing performance without sacrificing quality. Experimental results demonstrate that M3Prune outperforms both single-agent and existing multi-agent mRAG systems, achieving superior task performance while significantly improving token efficiency.
Reducing token overhead by pruning redundant communication edges allows multi-agent systems to achieve better performance without the computational burden.
Recent advances in multi-modal retrieval-augmented generation (mRAG), which augments multi-modal large language models (MLLMs) with external knowledge, have shown that collective intelligence from multiple agents can outperform a single model through effective communication. Despite their strong performance, existing multi-agent systems incur substantial token overhead and computational cost, posing challenges for large-scale deployment. To address these issues, we propose a Multi-Modal Multi-agent hierarchical communication graph PRUNING framework, termed M3Prune. M3Prune eliminates redundant communication edges both across and within modalities, improving the trade-off between task performance and token overhead. Specifically, M3Prune first performs intra-modal graph sparsification in the textual and visual modalities to identify task-critical communication links. It then constructs an inter-modal communication graph and sparsifies cross-modal connections while encouraging consistent cross-modal reasoning through a modality alignment score. Finally, it progressively prunes redundant edges to obtain an efficient hierarchical topology. Extensive experiments on both general-domain and domain-specific mRAG benchmarks show that M3Prune consistently outperforms single-agent and strong multi-agent mRAG systems while signifi- cantly improving token efficiency.
DAMO