Search papers, labs, and topics across Lattice.
This paper introduces LAG-Fusion, a latency-aware guidance fusion framework that enables asynchronous multimodal diffusion policy composition for robotic manipulation. By allowing modality-specific policies to operate at their native inference rates and providing denoising guidance when available, LAG-Fusion enhances responsiveness and performance in contact-rich tasks. Experimental results demonstrate that this approach outperforms both synchronous fusion methods and specially designed baselines, highlighting its effectiveness in managing heterogeneous modality latencies.
Asynchronous multimodal policies can significantly outperform traditional synchronous methods by leveraging native inference rates and dynamic guidance.
Diffusion policies have shown strong potential for robotic imitation learning, and recent extensions incorporate additional modalities to improve manipulation performance. However, these modalities often differ not only in information content but also in sensing rates and inference latencies. Existing multimodal diffusion policies typically rely on synchronous fusion or manually designed multi-frequency architectures, which either slow down high-frequency feedback or limit extensibility to new modality combinations. We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition. LAG-Fusion allows modality-specific policies to operate at their native inference rates and contribute denoising guidance whenever available. To make asynchronous composition consistent, we derive a reference-frame rebasing rule for diffusion variables under relative action representations, enabling delayed guidance to be aligned before fusion. We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy. Experiments under heterogeneous modality latencies show that LAG-Fusion improves policy responsiveness and task performance over synchronous fusion and specially designed force-aware baselines.
