HERCULES: An Open-Source Simulation Framework for Heterogeneous Multi-Robot SLAM, Collaborative Perception, and Exploration

2026-06-21

AI Summary

Existing simulation platforms struggle to support heterogeneous multi-robot systems—such as unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs)—in performing collaborative SLAM, perception, and exploration tasks within large-scale, high-fidelity dynamic environments. This work proposes an open-source simulation framework built upon Unreal Engine 5 that introduces a unified navigation and control architecture specifically designed for UAV–UGV heterogeneous collaboration. The framework features physically realistic long-wave infrared and night-vision sensor models and incorporates dynamic environmental elements such as fire and flooding. Extending AirSim/Cosys-AirSim, it integrates ROS 2 interfaces, a lightweight API, and strict time synchronization to support both replayable trajectories and online planning. The project also provides a benchmark dataset for heterogeneous multi-robot SLAM across desert, forest, and urban scenarios, demonstrating the system’s efficacy in collaborative mapping and exploration; all code and data are publicly released.

Abstract

We present HERCULES, an open-source simulator and data-collection pipeline for heterogeneous multi-robot autonomy. Built upon the Unreal Engine 5 (UE5)-based simulators AirSim and Cosys-AirSim, HERCULES resolves key architectural limitations of prior frameworks to enable concurrent unmanned aerial and ground vehicle (UAV-UGV) operation in large-scale, photorealistic, dynamic environments. It introduces a new waypoint-tracking UGV controller that mirrors existing UAV control interfaces, and provides a shared navigation stack for mapping, traversability analysis, planning, and control across heterogeneous platforms. Expanding inherited sensor suites, it adds physics-based long-wave infrared (LWIR) cameras and configurable night-vision modes for degraded visual environments. HERCULES provides lightweight APIs, ROS 2 wrappers, and rigorous time synchronization across sensors and platforms, and brings state-of-the-art game-engine capabilities into robotics simulation, integrating intelligent agents such as pedestrians, traffic, and wildlife with high-fidelity dynamic phenomena, including fire, flooding, and crop disease spread. HERCULES runs in two modes: passively, replaying offline-designed trajectories to generate reproducible multi-modal datasets, and actively, running an online planner in closed loop from live observations. Our experiments in heterogeneous multi-robot SLAM, collaborative perception, and exploration, using both HERCULES-generated data and active closed-loop execution, demonstrate its utility for advancing heterogeneous multi-robot autonomy. We publicly release our source code, experiment code, documentation, and datasets, including a heterogeneous multi-robot SLAM benchmark collected with two UAVs and two UGVs across kilometer-scale desert, forest, and city environments, at https://lunarlab-gatech.github.io/HERCULES-website.

Problem

• heterogeneous multi-robot
• SLAM
• collaborative perception
• exploration
• simulation framework

Innovation

• heterogeneous multi-robot
• photorealistic simulation
• LWIR sensing
• shared navigation stack
• closed-loop exploration