Reinforcement Learning-based Robust Wall Climbing Locomotion Controller in Ferromagnetic Environment

2025-10-22

AI Summary

To address locomotion instability in quadrupedal magnetic wall-climbing robots caused by uncertain foot adhesion on ferromagnetic surfaces—such as sensitivity to air gaps, partial contact, and stochastic detachment—this paper proposes a robust deep reinforcement learning (DRL) control framework. Methodologically: (i) a high-fidelity magnetic adhesion dynamics model is developed; (ii) a three-stage curriculum learning strategy—ground locomotion → gravity-induced rotation → detachment noise injection—is designed to progressively enhance disturbance rejection and recovery capability; and (iii) reliable sim-to-real transfer is achieved. Experiments demonstrate that the method significantly outperforms an MPC baseline assuming idealized adhesion in simulation, exhibiting rapid recovery from detachment events. Physical validation confirms stable, cable-free climbing on vertical steel walls. The core contribution lies in the first integration of realistic magnetic adhesion modeling with staged curriculum DRL, effectively resolving motion reliability challenges under weak or uncertain adhesion conditions.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We present a reinforcement learning framework for quadrupedal wall-climbing locomotion that explicitly addresses uncertainty in magnetic foot adhesion. A physics-based adhesion model of a quadrupedal magnetic climbing robot is incorporated into simulation to capture partial contact, air-gap sensitivity, and probabilistic attachment failures. To stabilize learning and enable reliable transfer, we design a three-phase curriculum: (1) acquire a crawl gait on flat ground without adhesion, (2) gradually rotate the gravity vector to vertical while activating the adhesion model, and (3) inject stochastic adhesion failures to encourage slip recovery. The learned policy achieves a high success rate, strong adhesion retention, and rapid recovery from detachment in simulation under degraded adhesion. Compared with a model predictive control (MPC) baseline that assumes perfect adhesion, our controller maintains locomotion when attachment is intermittently lost. Hardware experiments with the untethered robot further confirm robust vertical crawling on steel surfaces, maintaining stability despite transient misalignment and incomplete attachment. These results show that combining curriculum learning with realistic adhesion modeling provides a resilient sim-to-real framework for magnetic climbing robots in complex environments.

Problem

• Addresses uncertainty in magnetic foot adhesion for climbing robots
• Stabilizes learning through three-phase curriculum for reliable transfer
• Maintains locomotion during intermittent adhesion failures in complex environments

Innovation

• Reinforcement learning framework for quadrupedal wall climbing
• Three-phase curriculum learning with adhesion modeling
• Realistic adhesion simulation enabling robust sim-to-real transfer