🤖 AI Summary
This work addresses reward-free, offline, image-driven goal-oriented robotic manipulation—enabling end-to-end vision-based grasping and relocation of real-world objects using only a single target image. Methodologically, we propose a contrastive learning–based self-supervised offline reinforcement learning framework, integrating an image encoder–action decoder architecture with goal-conditioned policy learning; crucial architectural designs and hyperparameter configurations are introduced to ensure stable training for real-hardware deployment. To the best of our knowledge, this is the first demonstration of contrastive self-supervised RL on a physical robotic arm. Our approach achieves a twofold improvement in task success rate over baseline methods. Critically, it requires no handcrafted reward functions, online environment interaction, or pixel-level annotations—significantly lowering the barrier to real-world deployment.
📝 Abstract
Robotic systems that rely primarily on self-supervised learning have the potential to decrease the amount of human annotation and engineering effort required to learn control strategies. In the same way that prior robotic systems have leveraged self-supervised techniques from computer vision (CV) and natural language processing (NLP), our work builds on prior work showing that the reinforcement learning (RL) itself can be cast as a self-supervised problem: learning to reach any goal without human-specified rewards or labels. Despite the seeming appeal, little (if any) prior work has demonstrated how self-supervised RL methods can be practically deployed on robotic systems. By first studying a challenging simulated version of this task, we discover design decisions about architectures and hyperparameters that increase the success rate by $2 imes$. These findings lay the groundwork for our main result: we demonstrate that a self-supervised RL algorithm based on contrastive learning can solve real-world, image-based robotic manipulation tasks, with tasks being specified by a single goal image provided after training.
