PCLA: A Framework for Testing Autonomous Agents in the CARLA Simulator

2025-03-12

AI Summary

Reusing high-performance pretrained autonomous driving agents from the CARLA Leaderboard in simulation remains challenging due to tight coupling between agents and the CARLA environment, hindering flexible deployment across custom scenarios and CARLA versions. Method: This paper introduces the first open-source decoupled testing framework that fully separates Leaderboard agents from the CARLA simulator. It standardizes agent interaction via a lightweight Python API, modularizes scenario interfaces, and eliminates dependencies on specific CARLA versions or Leaderboard codebases. Contribution/Results: The framework enables plug-and-play deployment and seamless switching among nine state-of-the-art Leaderboard agents in arbitrary user-defined scenarios—without modifying original Leaderboard code or requiring version-specific adaptations. It ensures cross-version compatibility and zero-dependency integration. Validated on the latest CARLA release, the framework is publicly released to lower the barrier for autonomous driving algorithm evaluation and foster community-driven benchmarking.

Technology Category

• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Recent research on testing autonomous driving agents has grown significantly, especially in simulation environments. The CARLA simulator is often the preferred choice, and the autonomous agents from the CARLA Leaderboard challenge are regarded as the best-performing agents within this environment. However, researchers who test these agents, rather than training their own ones from scratch, often face challenges in utilizing them within customized test environments and scenarios. To address these challenges, we introduce PCLA (Pretrained CARLA Leaderboard Agents), an open-source Python testing framework that includes nine high-performing pre-trained autonomous agents from the Leaderboard challenges. PCLA is the first infrastructure specifically designed for testing various autonomous agents in arbitrary CARLA environments/scenarios. PCLA provides a simple way to deploy Leaderboard agents onto a vehicle without relying on the Leaderboard codebase, it allows researchers to easily switch between agents without requiring modifications to CARLA versions or programming environments, and it is fully compatible with the latest version of CARLA while remaining independent of the Leaderboard's specific CARLA version. PCLA is publicly accessible at https://github.com/MasoudJTehrani/PCLA.

Problem

• Challenges in testing autonomous agents in customized CARLA environments.
• Difficulty in deploying Leaderboard agents without Leaderboard codebase dependency.
• Need for a framework to test various agents in arbitrary CARLA scenarios.

Innovation

• Open-source Python framework for testing autonomous agents
• Includes nine pre-trained high-performing CARLA agents
• Enables easy deployment and switching between agents