Search papers, labs, and topics across Lattice.
This paper introduces FLARE, a framework designed to enhance Vision-Language-Action Models (VLAs) by enabling them to autonomously recover from execution errors during robotic manipulation tasks. By incorporating a "Retry" mechanism that utilizes perturbations and bridging segments, FLARE allows VLAs to handle deviations in execution, while a "Reset" pipeline leverages a multi-layered language model (MLLM) for offline failure analysis to identify and address out-of-distribution (OOD) states. Experimental results demonstrate that FLARE significantly improves both task success rates and robustness in complex, contact-rich environments.
FLARE transforms VLAs from brittle performers into resilient agents capable of autonomously recovering from common execution failures in robotic manipulation.
Vision-Language-Action Models~(VLAs) have demonstrated significant promise in generalizing to complex, long-horizon robotic manipulation tasks. However, their performance remains brittle, as they are typically trained on trajectory-monotonic, failure-free demonstrations. This reliance on ``perfect"data leaves them unable to recover from common execution errors, such as a missed grasp, a dropped object, or an unexpected collision. In this paper, we propose FLARE, a novel framework that endows VLAs with robust error recovery capabilities through a ``Retry"and ``Reset"paradigm. First, we introduce a ``Retry"mechanism by injecting perturbation and bridging segments that decouple robot pose from environment state into demonstrations, enabling the policy to autonomously handle execution deviations. Second, to address critical, state-breaking (OOD) failures, we introduce a ``Reset"pipeline. We leverage an MLLM for offline failure analysis to automatically identify OOD states from execution videos. This analysis enables the efficient, targeted collection of a small library of object-centric ``Reset"skills, which are trained to restore the environment to a task-valid state. Our full framework integrates these learned policies. At inference, an online MLLM monitor arbitrates between task execution and ``Reset"skills. Experiments on challenging, contact-rich manipulation tasks show our approach significantly improves task success and robustness.
