Search papers, labs, and topics across Lattice.
The paper introduces Pose6DAug, a failure-driven data augmentation framework that leverages successful episodes from a vision-language-action (VLA) policy to create targeted demonstrations for failure modes without requiring new data collection. By swapping manipulated objects while maintaining the original action trajectory and ensuring multi-view consistency through a 3D approach, the method addresses the challenges of occlusion and viewpoint changes. The results demonstrate a 16.5% relative improvement in success rates on novel objects compared to state-of-the-art methods, while maintaining performance on in-distribution tasks.
Swapping objects in 3D space can boost VLA policy success rates by over 16% on novel objects without additional data collection.
Vision-language-action (VLA) policies have shown strong potential for general-purpose manipulation, yet they often fail on novel, out-of-distribution objects whose appearance or geometry deviates from the training distribution. The standard remedy is to collect multi-view teleoperation data for every failure case, but this scales poorly in both cost and time. We introduce Pose6DAug, a failure-driven data augmentation framework that turns a policy's own successful episodes into targeted demonstrations for its failure modes, without any new data collection. Our key insight is that each successful episode already encodes a physically valid action trajectory together with calibrated multi-view observations. By swapping only the manipulated object while preserving this trajectory, we obtain new and physically grounded demonstrations. However, naive 2D video editing breaks multi-view consistency and physical plausibility, particularly under heavy occlusion and egocentric viewpoints. Our method instead operates directly in 3D, anchoring the target object with an explicit mesh driven by a temporally coherent 6D pose trajectory, ensuring geometrically consistent renderings across all camera views. Fine-tuning a VLA on data augmented by our method improves success rates by 16.5% relative to the state-of-the-art baseline on novel objects, while preserving in-distribution performance. These results show that multi-view and physically consistent augmentation is a practical path to scalable VLA generalization.
