Search papers, labs, and topics across Lattice.
This paper introduces a multimodal multi-agent framework for automatic workflow execution that learns a topological knowledge base from fragmented execution logs during an offline discovery phase. During inference, agents use adaptive RAG over this graph, along with a closed-loop collaborative verification protocol for self-correction and navigation. Experiments in a real-world context demonstrate the framework's ability to maintain high reliability and semantic awareness even with limited training data.
Fragmented execution logs can be stitched together into a topological knowledge base that allows multi-agent systems to execute workflows more reliably and with greater semantic awareness.
Modern information systems require autonomous agents capable of navigating complex workflows, yet current methodologies often struggle with the transition from structured metadata parsing to general environmental perception. While the integration of MLLMs has enabled agents to interact directly with GUIs, existing approaches typically treat task sequences as discrete, linear episodes. This fragmentation prevents agents from capturing the underlying transition topology, limiting their effectiveness in novel or non-stationary scenarios. To address this, we propose a novel multimodal multi-agent framework that achieves automatic workflow execution through a distinct two-phase pipeline. First, during an offline discovery phase, the architecture adaptively constructs a topological knowledge base from fragmented execution logs. During inference, agents leverage Adaptive Retrieval-Augmented Generation (RAG) over this fixed, pre-established graph, coupled with a closed-loop collaborative verification protocol to dynamically self-correct and navigate. This graph-based approach facilitates superior task decomposition and adaptive navigation performance. We validate our framework in a real-world context, demonstrating its ability to maintain high reliability and semantic awareness even with limited training data.
