🤖 AI Summary
This work addresses the challenge of general-purpose, distributed haptic manipulation of desktop objects using only tactile feedback. Method: We propose ArrayBot—a programmable 16×16 tactile pin array enabling object support, perception, and multimodal manipulation (relocation, pushing, and herding) solely through haptic signals. We introduce two key innovations: (i) spatially localized action block modeling to handle high-dimensional, redundant actuation spaces, and (ii) a frequency-domain low-frequency action reshaping mechanism to improve control stability and generalization. These are integrated with end-to-end reinforcement learning (PPO/SAC). Contribution/Results: The learned policies generalize across unseen object geometries in simulation and achieve zero-shot, no-fine-tuning sim-to-real transfer on physical hardware, successfully executing diverse manipulation tasks. This work establishes a scalable, transferable paradigm for tactile-driven general physical manipulation.
📝 Abstract
We present ArrayBot, a distributed manipulation system consisting of a 16 × 16 array of vertically sliding pillars integrated with tactile sensors. Functionally, ArrayBot is designed to simultaneously support, perceive, and manipulate the tabletop objects. Towards generalizable distributed manipulation, we leverage reinforcement learning (RL) algorithms for the automatic discovery of control policies. In the face of the massively redundant actions, we propose to reshape the action space by considering the spatially local action patch and the low-frequency actions in the frequency domain. With this reshaped action space, we train RL agents that can relocate diverse objects through tactile observations only. Intriguingly, we find that the discovered policy can not only generalize to unseen object shapes in the simulator but also have the ability to transfer to the physical robot without any sim-to-real fine-tuning. |Leveraging the deployed policy, we derive more real-world manipulation skills on ArrayBot to further illustrate the distinctive merits of our proposed system.
