Search papers, labs, and topics across Lattice.
This paper introduces RoboSaGA, a Saliency-Guided Augmentation method designed to enhance the robustness of vision-based behavior cloning (BC) against significant visual domain shifts. By leveraging policy-driven saliency to dynamically adjust augmentation intensity at the pixel level, RoboSaGA effectively preserves critical task information while allowing for aggressive augmentation in less relevant areas. Experimental results demonstrate that RoboSaGA maintains in-domain performance while significantly improving resilience to variations in distractors, backgrounds, and lighting conditions in both simulated and real-world environments.
Saliency-guided augmentation can drastically improve the robustness of behavior cloning models to visual domain shifts without sacrificing in-domain performance.
In vision-based behavior cloning (BC), conventional image augmentations such as Random Crop and Color Jitter often fall short under substantial visual domain shifts, including changes in shadows, distractors, and backgrounds. Superimposition-based augmentations, which blend in-domain and out-of-domain images, have shown promise for improving generalization in computer vision, but their suitability for BC remains uncertain because task-critical semantics, spatiotemporal relationships, and agent-target interactions must be preserved. To address this, we introduce RoboSaGA, a Saliency-Guided Augmentation method within the superimposition family tailored for vision-based BC. RoboSaGA dynamically adjusts augmentation intensity at the pixel level using policy-driven saliency, enabling aggressive augmentation in task-irrelevant regions while preserving task-critical information. It integrates seamlessly into existing architectures without requiring structural modifications or additional learning objectives. Experiments in both simulated and real-world settings show that RoboSaGA preserves in-domain performance while substantially improving robustness to visual domain shifts, including distractor and background changes, as well as lighting and shadow variations. Code is available at https://github.com/Zheyu-Zhuang/RoboSaGA.
