Deep RL- Tuned Mo del-Free Adaptive Control for Lower-Limb Exoskeletons During Sit-to-Stand Transitions

2026-06-20

AI Summary

This study addresses the challenge of trajectory tracking during sit-to-stand transitions in lower-limb exoskeletons, which is complicated by complex, time-varying human–robot interaction and inter-subject variability. To this end, an intelligent model-free adaptive backstepping control strategy is proposed. The approach integrates a second-order ultra-local model with a radial basis function (RBF) neural network for online estimation of unknown dynamics and, for the first time, employs a Twin Delayed Deep Deterministic Policy Gradient (TD3) reinforcement learning agent as a phase-aware gain scheduler to achieve adaptive gain tuning without system identification. Simulation results demonstrate that the proposed method achieves an average joint tracking RMSE of 0.078°, representing improvements of 60.2%, 54.4%, 48.7%, and 42.6% over PID, MFAC, LQR, and SMC baselines, respectively. Moreover, integrating TD3 further reduces hip, knee, and ankle tracking errors by 35%, 33%, and 79%, substantially enhancing both accuracy and robustness.

Abstract

Sit-to-stand (STS) transitions impose significant joint-loading demands on elderly individuals, making them a primary target for lower-limb exoskeleton assistance. However, accurate trajectory tracking during STS is challenging due to complex, time-varying human exoskeleton interaction dynamics and inter-subject variability that render model-based control approaches difficult to apply in practice. This paper presents an intelligent model free adaptive backstepping control strategy for a bilateral lower-limb exoskeleton during STS motion. The proposed controller design uses an ultra-local second-order model to avoid explicit system identification, while a Gaussian radial basis function (RBF) neural network estimates the unknown lumped dynamics online. To further improve phase-aware tracking performance, a Twin Delayed Deep Deterministic Policy Gradient (TD3) reinforcement learning agent is integrated as a supervisory gain scheduler that adaptively adjusts controller gains across the distinct phases of STS motion. The proposed controller is evaluated through co-simulation in MATLAB/Simulink and Simscape Multibody using OpenSim-derived reference trajectories and benchmarked against state-of-the-art controllers. Results demonstrate that the proposed controller achieves the lowest average RMSE of 0.078 degree across all joints, representing improvements of 60.2%, 54.4%, 48.7%, and 42.6% over proportional integral derivative (PID), model-free adaptive control (MFAC), linear quadratic regulator (LQR), and sliding-mode control (SMC), respectively. TD3 integration further reduces tracking error by 35%, 33%, and 79% at the hip, knee, and ankle joints compared to the standalone RBF-MFAC baseline. These results demonstrate the effectiveness and robustness of the proposed controller design for assistive exoskeleton control during STS transitions.

Problem

• sit-to-stand transitions
• trajectory tracking
• human-exoskeleton interaction
• inter-subject variability
• model-based control

Innovation

• model-free adaptive control
• deep reinforcement learning
• sit-to-stand transition
• RBF neural network
• TD3