RSL-RL: A Learning Library for Robotics Research

2025-09-12

AI Summary

General-purpose reinforcement learning (RL) frameworks struggle to meet robotics research requirements in algorithm adaptability, computational efficiency, and hardware co-design. This paper introduces RoboRL—a lightweight, open-source RL library tailored for robotic control—built upon a minimalist design philosophy. It supports end-to-end GPU training, seamless sim-to-real transfer, and modular algorithm extension. Its core contributions include domain-specific auxiliary techniques embedded to address robotics challenges: accurate dynamics modeling, sparse reward settings, and safety-critical constraints. Furthermore, its highly compact and readable codebase drastically reduces integration overhead for novel algorithms. Extensive experiments demonstrate that RoboRL achieves high-throughput training and robust deployment across diverse simulation benchmarks (e.g., Isaac Gym) and real-world robotic platforms (e.g., Unitree A1). By unifying performance, extensibility, and practicality, RoboRL establishes a new paradigm for developing learning-based controllers in robotics.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

RSL-RL is an open-source Reinforcement Learning library tailored to the specific needs of the robotics community. Unlike broad general-purpose frameworks, its design philosophy prioritizes a compact and easily modifiable codebase, allowing researchers to adapt and extend algorithms with minimal overhead. The library focuses on algorithms most widely adopted in robotics, together with auxiliary techniques that address robotics-specific challenges. Optimized for GPU-only training, RSL-RL achieves high-throughput performance in large-scale simulation environments. Its effectiveness has been validated in both simulation benchmarks and in real-world robotic experiments, demonstrating its utility as a lightweight, extensible, and practical framework to develop learning-based robotic controllers. The library is open-sourced at: https://github.com/leggedrobotics/rsl_rl.

Problem

• Developing a specialized reinforcement learning library for robotics
• Addressing robotics-specific challenges with optimized algorithms
• Enabling efficient and extensible learning-based robotic controllers

Innovation

• Open-source reinforcement learning library
• Compact modifiable codebase for robotics
• GPU-optimized high-throughput training performance