Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To address the catastrophic failure of vision-language-action (VLA) models when visual feeds drop mid-execution, the authors establish MAIL-Bench to systematically evaluate policy degradation under dynamic camera loss. They develop MINT, a framework that trains policies to tolerate missing modalities and uses action-conditioned world models or optical flow to synthesize missing observations at inference time. Evaluated on $\pi_{0.5}$, GR00T N1.5, and real AgiBot G2 hardware, MINT substantially recovers manipulation success rates under multi-stage visual interruptions by deploying and un-deploying predicted visual rollouts based on reliability.
Losing a single camera feed derails state-of-the-art VLA models, but pairing action-conditioned world model extrapolation with selective view withdrawal keeps closed-loop manipulation intact during sensor failures.
Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, but they are typically developed and evaluated with all camera streams available throughout task execution. When a camera stops delivering frames during task execution, the policy must continue acting without access to subsequent observations from the missing view. Despite its practical importance, how such interruptions affect closed-loop manipulation remains insufficiently understood. To investigate this problem, we introduce MAIL-Bench, a benchmark that evaluates visual interruptions with VLA models. By interrupting different cameras at multiple stages of each policy's successful reference trajectory, MAIL-Bench measures how well policies retain their capabilities when visual inputs become unavailable. Building on this benchmark, we propose MINT, which first trains VLA policies to remain functional under missing visual inputs. At inference time, MINT selectively supplements missing observations using optical-flow extrapolation or an action-conditioned world model, and withdraws predicted views when they become unreliable. Experiments on $π_{0.5}$ and GR00T N1.5 show that MINT significantly improves task success under camera loss over the original models. Experiments on AgiBot G2 further demonstrate the real-robot deployment under camera loss. The benchmark is available at https://minglejiang.github.io/Mail-Bench/