Enhancing Visual Domain Robustness in Behaviour Cloning via Saliency-Guided Augmentation

2026-08-12

AI Summary

This work addresses the limited generalization of conventional image augmentation methods in visual behavior cloning when confronted with visual domain shifts such as shadows, distractors, and background variations. To overcome this challenge, the authors propose RoboSaGA—a saliency-guided overlay augmentation approach that, for the first time, integrates policy-driven saliency maps into data augmentation. By dynamically modulating augmentation intensity at the pixel level, RoboSaGA applies strong perturbations to task-irrelevant regions while preserving critical task-related areas. Notably, this method achieves task-aware adaptive augmentation without modifying the network architecture or introducing auxiliary learning objectives. Experimental results demonstrate that RoboSaGA maintains original in-domain performance while significantly improving robustness against diverse visual domain shifts in both simulated and real-world environments.

Abstract

In vision-based behavior cloning (BC), conventional image augmentations such as Random Crop and Color Jitter often fall short under substantial visual domain shifts, including changes in shadows, distractors, and backgrounds. Superimposition-based augmentations, which blend in-domain and out-of-domain images, have shown promise for improving generalization in computer vision, but their suitability for BC remains uncertain because task-critical semantics, spatiotemporal relationships, and agent-target interactions must be preserved. To address this, we introduce RoboSaGA, a Saliency-Guided Augmentation method within the superimposition family tailored for vision-based BC. RoboSaGA dynamically adjusts augmentation intensity at the pixel level using policy-driven saliency, enabling aggressive augmentation in task-irrelevant regions while preserving task-critical information. It integrates seamlessly into existing architectures without requiring structural modifications or additional learning objectives. Experiments in both simulated and real-world settings show that RoboSaGA preserves in-domain performance while substantially improving robustness to visual domain shifts, including distractor and background changes, as well as lighting and shadow variations. Code is available at https://github.com/Zheyu-Zhuang/RoboSaGA.

Problem

• visual domain shift
• behavior cloning
• image augmentation
• task-critical semantics
• domain robustness

Innovation

• Saliency-Guided Augmentation
• Behavior Cloning
• Visual Domain Robustness
• Superimposition-based Augmentation
• Policy-driven Saliency