Experimental Setup and Software Pipeline to Evaluate Optimization based Autonomous Multi-Robot Search Algorithms

2025-06-20

AI Summary

Most multi-robot search algorithms are validated solely in simulation, lacking rigorous evaluation in real-world physical environments—particularly for signal source localization—thus suffering from a significant sim-to-real gap. Method: We develop a reproducible lab-scale hardware-software platform to bridge this gap. Our approach introduces (i) a low-cost, controllable acoustic source synchronized with e-puck robots within a motion-capture system; (ii) an asynchronous, plug-and-play distributed algorithm interface middleware; and (iii) explicit SNR modeling to quantify simulation–reality discrepancies. Built upon ROS, Vicon, and Swarm/Bayesian optimization frameworks, we empirically benchmark Swarm optimization, Bayes-Swarm, and random walk algorithms on physical hardware. Contribution/Results: This work presents the first quantitative analysis—on real robots—of the trade-off between search performance and real-time computational overhead. All code and experimental protocols are publicly released.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Localization, Mapping, and Navigation
• Constraint Satisfaction and Optimization: Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Signal source localization has been a problem of interest in the multi-robot systems domain given its applications in search &rescue and hazard localization in various industrial and outdoor settings. A variety of multi-robot search algorithms exist that usually formulate and solve the associated autonomous motion planning problem as a heuristic model-free or belief model-based optimization process. Most of these algorithms however remains tested only in simulation, thereby losing the opportunity to generate knowledge about how such algorithms would compare/contrast in a real physical setting in terms of search performance and real-time computing performance. To address this gap, this paper presents a new lab-scale physical setup and associated open-source software pipeline to evaluate and benchmark multi-robot search algorithms. The presented physical setup innovatively uses an acoustic source (that is safe and inexpensive) and small ground robots (e-pucks) operating in a standard motion-capture environment. This setup can be easily recreated and used by most robotics researchers. The acoustic source also presents interesting uncertainty in terms of its noise-to-signal ratio, which is useful to assess sim-to-real gaps. The overall software pipeline is designed to readily interface with any multi-robot search algorithm with minimal effort and is executable in parallel asynchronous form. This pipeline includes a framework for distributed implementation of multi-robot or swarm search algorithms, integrated with a ROS (Robotics Operating System)-based software stack for motion capture supported localization. The utility of this novel setup is demonstrated by using it to evaluate two state-of-the-art multi-robot search algorithms, based on swarm optimization and batch-Bayesian Optimization (called Bayes-Swarm), as well as a random walk baseline.

Problem

• Evaluating multi-robot search algorithms in real-world settings
• Developing lab-scale setup for acoustic source localization
• Comparing algorithm performance in physical vs simulated environments

Innovation

• Lab-scale acoustic source and e-puck robots
• Open-source ROS-based software pipeline
• Parallel asynchronous algorithm execution