Residual Reinforcement Learning for Robot Teleoperation under Stochastic Delays

2026-05-14

AI Summary

This work addresses the challenge of discontinuous teleoperation signals caused by stochastic communication delays, which induce control instability and high-frequency jitter—issues poorly handled by conventional reinforcement learning approaches. To overcome this, the authors propose a delay-robust hybrid control framework that uniquely integrates an LSTM-based state estimator with a residual reinforcement learning policy. The LSTM module reconstructs continuous system states from delayed observations, while the residual policy learns compensatory torques that jointly optimize trajectory tracking accuracy and joint velocity smoothness. Evaluated on a Franka Panda robotic platform, the proposed method demonstrates superior performance over existing techniques, maintaining stable and smooth teleoperation even under high-variance random delays.

Abstract

Stochastic communication delays in teleoperation introduce signal discontinuities that undermine control stability and degrade control performance. Consequently, the conventional reinforcement learning (RL) methods struggle with the delayed observations due to the delay-induced observations, leading to high-frequency chattering. To address this, we propose a hybrid control framework, delay-resilient RL, integrating a state estimator utilizing Long Short-Term Memory (LSTM) with a residual RL policy, which is resilient to stochastic delays. The LSTM reconstructs smooth, continuous state estimates from delayed observations, enabling the RL agent to learn a residual torque compensation policy that balances tracking accuracy with velocity smoothness. Experimental validation on Franka Panda robots demonstrates that our approach significantly outperforms the state-of-the-art baselines, ensuring robust and stable teleoperation even under high-variance stochastic delays.

Problem

• teleoperation
• stochastic delays
• reinforcement learning
• control stability
• signal discontinuities

Innovation

• residual reinforcement learning
• stochastic delays
• LSTM-based state estimation
• teleoperation
• delay-resilient control