Enhancing Tactile-based Reinforcement Learning for Robotic Control

2025-10-24

AI Summary

Robust robotic tactile manipulation is hindered by limited visual perception and reliance on idealized state information. Method: We propose a decoupled tactile reinforcement learning framework that separates self-supervised learning (SSL) for tactile-proprioceptive feature extraction and memory modeling from online RL policy optimization. This architecture is the first to empirically validate the sufficiency and critical role of sparse binary tactile signals in decoupled motor control. Contribution/Results: To advance standardization, we introduce and open-source the Robot Tactile Olympiad (RoTO) benchmark. Experiments demonstrate human-surpassing performance on challenging dexterous tasks—including ball bouncing and Baoding ball rotation—while significantly improving generalization, safety, and dexterity of tactile-driven control. Our approach establishes a new paradigm for low-bandwidth, high-robustness tactile agents in real-world settings.

Technology Category

• Intelligent Robots: Manipulation
• Natural Language Processing: Safety and Robustness
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Achieving safe, reliable real-world robotic manipulation requires agents to evolve beyond vision and incorporate tactile sensing to overcome sensory deficits and reliance on idealised state information. Despite its potential, the efficacy of tactile sensing in reinforcement learning (RL) remains inconsistent. We address this by developing self-supervised learning (SSL) methodologies to more effectively harness tactile observations, focusing on a scalable setup of proprioception and sparse binary contacts. We empirically demonstrate that sparse binary tactile signals are critical for dexterity, particularly for interactions that proprioceptive control errors do not register, such as decoupled robot-object motions. Our agents achieve superhuman dexterity in complex contact tasks (ball bouncing and Baoding ball rotation). Furthermore, we find that decoupling the SSL memory from the on-policy memory can improve performance. We release the Robot Tactile Olympiad (RoTO) benchmark to standardise and promote future research in tactile-based manipulation. Project page: https://elle-miller.github.io/tactile_rl

Problem

• Enhancing tactile reinforcement learning for safe robotic manipulation
• Overcoming inconsistent tactile sensing efficacy in robot control
• Developing self-supervised methods for sparse tactile observations

Innovation

• Self-supervised learning enhances tactile reinforcement learning
• Sparse binary tactile signals enable dexterous robot manipulation
• Decoupling SSL memory from on-policy memory improves performance