Search papers, labs, and topics across Lattice.
This paper introduces the Dynamic Agent-based Interaction Network (DAIN), a novel framework for multimodal fusion that transforms the process into a dynamic, collaborative interaction among specialized agents. By utilizing a context-aware Meta-Controller, DAIN optimizes the scheduling of agent activations and facilitates efficient inter-agent communication, achieving a notable 2.6% accuracy improvement on the ADNI benchmark. Comprehensive evaluations across five diverse datasets confirm DAIN's status as a new state-of-the-art solution in multimodal reasoning, highlighting its operational efficiency and enhanced interpretability.
Dynamic agent collaboration in DAIN leads to a 2.6% accuracy boost on multimodal tasks, redefining efficiency in complex reasoning scenarios.
Current multimodal fusion approaches, particularly those based on static Mixture-of-Experts (MoE) architectures, often struggle to provide the adaptive and efficient collaborative reasoning required by complex real-world applications. We introduce the Dynamic Agent-based Interaction Network (DAIN), which reconceptualizes multimodal fusion as a dynamic, multi-agent collaborative process. DAIN employs a context-aware Meta-Controller that dynamically schedules sparse activation of specialized interaction agents and orchestrates compressed inter-agent communication for consensus-building. The framework is guided by a multi-objective loss function that jointly optimizes task accuracy, agent specialization, and operational efficiency through sparse activation and communication regularization. Comprehensive evaluations across five diverse benchmarks -- ADNI, MIMIC-IV, MM-IMDB, CMU-MOSI, and ENRICO -- establish DAIN as a new state-of-the-art, delivering significant performance improvements including a 2.6\% accuracy gain on ADNI. Ablation studies verify the critical roles of both dynamic scheduling and agent communication. Furthermore, DAIN offers enhanced interpretability by exposing context-dependent agent roles and collaboration patterns while maintaining computational efficiency through sample-wise sparse agent activation. Our work demonstrates the promise of dynamic, agent-based paradigms for multimodal reasoning.
