Search papers, labs, and topics across Lattice.
This paper introduces FoMo-FD, a novel failure detection method for surgical robot imitation policies that utilizes an action-conditioned flow-matching world model to learn nominal short-horizon visual dynamics. By scoring the inverse-transport nonconformity of observed endpoint latents, FoMo-FD effectively detects visual-action inconsistencies without needing failure demonstrations, thus addressing the challenges posed by scarce failure data and high variability in surgical tasks. Evaluated across four manipulation tasks with twenty failure modes, FoMo-FD achieves a remarkable 96.6% failure detection rate at a mere 1.3% false alarm rate, significantly outperforming existing anomaly detection baselines.
FoMo-FD achieves a 96.6% failure detection rate in surgical tasks without requiring prior failure data, setting a new standard for safety in robotic surgery.
Imitation learning has shown increasing promise for autonomous robotic surgery, yet safe deployment remains challenging due to the safety-critical nature of surgical tasks and the complexity and variability of surgical environments. Failure detection is therefore an essential safeguard, but its development remains difficult due to the challenges of scarce failure data, highly variable manipulation dynamics, and the need to balance missed detections against disruptive false alarms. To address these challenges, we introduce FoMo-FD (Flow-Matching World Model for Failure Detection), a failure detection method that learns nominal short-horizon visual dynamics with an action-conditioned flow-matching world model. FoMo-FD scores the inverse-transport nonconformity of observed endpoint latents, enabling window-level detection of visual-action inconsistencies without requiring failure demonstrations. Detection thresholds are obtained by conformal calibration on successful executions, yielding task-specific alarms without assuming future failure types. We evaluate FoMo-FD on four surgically relevant manipulation tasks with twenty failure modes across simulation and real-world experiments using the da Vinci Research Kit (dVRK). Results show that FoMo-FD outperforms observation-level anomaly baselines and a prediction-error variant of the same world model, with the wrist-camera view achieving the strongest performance, including a 96.6% failure detection rate (FDR) at a 1.3% false alarm rate (FAR).
