Residual Rotation Correction using Tactile Equivariance

2025-11-10

AI Summary

To address low sample efficiency and poor generalization in vision–tactile policy learning, this paper proposes a tactile-equivariant residual rotation correction method. We explicitly embed SO(2) equivariance into the policy learning framework for the first time, designing a lightweight, symmetry-aware module that estimates and compensates for object orientation deviations in real time—without requiring additional human demonstrations. Our approach reconstructs surface normal fields from RGB images and employs an SO(2)-equivariant network to directly predict residual rotational actions, thereby enhancing a base visuomotor policy. Evaluated on a real robotic platform, the method achieves zero-shot generalization to unseen object orientations using only a small number of training samples. It significantly improves manipulation accuracy and robustness, outperforming existing baseline methods across key metrics.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Low Level & Physics-based Vision
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Visuotactile policy learning augments vision-only policies with tactile input, facilitating contact-rich manipulation. However, the high cost of tactile data collection makes sample efficiency the key requirement for developing visuotactile policies. We present EquiTac, a framework that exploits the inherent SO(2) symmetry of in-hand object rotation to improve sample efficiency and generalization for visuotactile policy learning. EquiTac first reconstructs surface normals from raw RGB inputs of vision-based tactile sensors, so rotations of the normal vector field correspond to in-hand object rotations. An SO(2)-equivariant network then predicts a residual rotation action that augments a base visuomotor policy at test time, enabling real-time rotation correction without additional reorientation demonstrations. On a real robot, EquiTac accurately achieves robust zero-shot generalization to unseen in-hand orientations with very few training samples, where baselines fail even with more training data. To our knowledge, this is the first tactile learning method to explicitly encode tactile equivariance for policy learning, yielding a lightweight, symmetry-aware module that improves reliability in contact-rich tasks.

Problem

• Improves sample efficiency in visuotactile policy learning
• Enables real-time rotation correction without demonstrations
• Achieves zero-shot generalization to unseen orientations

Innovation

• EquiTac framework exploits SO(2) symmetry
• Reconstructs surface normals from RGB tactile inputs
• Uses equivariant network for residual rotation correction