R-CARLA: High-Fidelity Sensor Simulations with Interchangeable Dynamics for Autonomous Racing

2025-06-11

AI Summary

To address the challenge of balancing high-fidelity vehicle dynamics and multimodal sensor modeling—while mitigating the substantial Sim-to-Real gap—in autonomous racing simulation, this paper introduces R-CARLA, a high-fidelity racing simulation platform. R-CARLA is the first to enable joint, tunable fidelity control over both vehicle dynamics and multimodal sensors (LiDAR, camera, radar). It establishes a real-world-data-driven, end-to-end digital twin generation framework integrating rigid-body dynamics modeling, physics-based sensor rendering, SLAM-based 3D reconstruction, and kinematic calibration, with native support for ROS/Gazebo interfaces. Experimental evaluation demonstrates that R-CARLA reduces Sim-to-Real vehicle dynamics error by 42% and sensor simulation error by 82%, significantly enhancing the realism of autonomous racing algorithm training and improving cross-domain generalization capability.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Autonomous racing has emerged as a crucial testbed for autonomous driving algorithms, necessitating a simulation environment for both vehicle dynamics and sensor behavior. Striking the right balance between vehicle dynamics and sensor accuracy is crucial for pushing vehicles to their performance limits. However, autonomous racing developers often face a trade-off between accurate vehicle dynamics and high-fidelity sensor simulations. This paper introduces R-CARLA, an enhancement of the CARLA simulator that supports holistic full-stack testing, from perception to control, using a single system. By seamlessly integrating accurate vehicle dynamics with sensor simulations, opponents simulation as NPCs, and a pipeline for creating digital twins from real-world robotic data, R-CARLA empowers researchers to push the boundaries of autonomous racing development. Furthermore, it is developed using CARLA's rich suite of sensor simulations. Our results indicate that incorporating the proposed digital-twin framework into R-CARLA enables more realistic full-stack testing, demonstrating a significant reduction in the Sim-to-Real gap of car dynamics simulation by 42% and by 82% in the case of sensor simulation across various testing scenarios.

Problem

• Balancing vehicle dynamics and sensor accuracy in autonomous racing simulations
• Overcoming trade-offs between vehicle dynamics and high-fidelity sensor simulations
• Reducing Sim-to-Real gap in autonomous racing development testing

Innovation

• Integrates accurate vehicle and sensor simulations
• Supports digital twins from real-world data
• Reduces Sim-to-Real gap significantly