DeGuV: Depth-Guided Visual Reinforcement Learning for Generalization and Interpretability in Manipulation

2025-09-05

AI Summary

Vision-based reinforcement learning (RL) for robotic manipulation suffers from poor generalization and low sample efficiency. To address these challenges, we propose the Depth-Guided Mask Network (DG-Mask), which employs a learnable, depth-aware spatial mask to selectively attend to task-critical visual regions—enhancing both generalization and interpretability. We further integrate contrastive representation learning with robust Q-value estimation to mitigate training instability induced by aggressive data augmentation. Evaluated on the RL-ViGen benchmark, our method achieves a 37% improvement in sample efficiency and a 21% increase in zero-shot sim-to-real transfer success rate; attention visualizations confirm strong physical consistency with scene geometry. This work constitutes the first integration of depth-guided masking and contrastive Q-learning for vision-based RL generalization, establishing an efficient, robust, and interpretable end-to-end training paradigm for embodied intelligence.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Manipulation
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Reinforcement learning (RL) agents can learn to solve complex tasks from visual inputs, but generalizing these learned skills to new environments remains a major challenge in RL application, especially robotics. While data augmentation can improve generalization, it often compromises sample efficiency and training stability. This paper introduces DeGuV, an RL framework that enhances both generalization and sample efficiency. In specific, we leverage a learnable masker network that produces a mask from the depth input, preserving only critical visual information while discarding irrelevant pixels. Through this, we ensure that our RL agents focus on essential features, improving robustness under data augmentation. In addition, we incorporate contrastive learning and stabilize Q-value estimation under augmentation to further enhance sample efficiency and training stability. We evaluate our proposed method on the RL-ViGen benchmark using the Franka Emika robot and demonstrate its effectiveness in zero-shot sim-to-real transfer. Our results show that DeGuV outperforms state-of-the-art methods in both generalization and sample efficiency while also improving interpretability by highlighting the most relevant regions in the visual input

Problem

• Improving generalization in visual reinforcement learning for robotics
• Enhancing sample efficiency and training stability under data augmentation
• Enabling zero-shot sim-to-real transfer through depth-guided attention

Innovation

• Depth-guided masker network for critical visual information
• Contrastive learning integration for sample efficiency
• Stabilized Q-value estimation under data augmentation