Search papers, labs, and topics across Lattice.
This paper introduces Performance Predictive Guidance (PPGuide), a classifier-based framework for improving the robustness of pre-trained diffusion policies in robotic manipulation. PPGuide uses attention-based multiple instance learning to self-supervise the identification of observation-action chunks relevant to task success or failure, enabling the training of a performance predictor. During inference, the performance predictor provides gradients to steer the diffusion policy away from failure modes, leading to improved performance on Robomimic and MimicGen benchmarks.
Steer your robot's diffusion policy away from failure modes at inference time with a lightweight performance predictor trained via self-supervised attention.
Diffusion policies have shown to be very efficient at learning complex, multi-modal behaviors for robotic manipulation. However, errors in generated action sequences can compound over time which can potentially lead to failure. Some approaches mitigate this by augmenting datasets with expert demonstrations or learning predictive world models which might be computationally expensive. We introduce Performance Predictive Guidance (PPGuide), a lightweight, classifier-based framework that steers a pre-trained diffusion policy away from failure modes at inference time. PPGuide makes use of a novel self-supervised process: it uses attention-based multiple instance learning to automatically estimate which observation-action chunks from the policy's rollouts are relevant to success or failure. We then train a performance predictor on this self-labeled data. During inference, this predictor provides a real-time gradient to guide the policy toward more robust actions. We validated our proposed PPGuide across a diverse set of tasks from the Robomimic and MimicGen benchmarks, demonstrating consistent improvements in performance.
