Before Autonomy Takes Control: Software Testing in Robotics

2026-02-02

AI Summary

This study addresses the significant challenges in testing robotic software, which arise from its tight coupling with hardware, high environmental uncertainty, and strong autonomy—factors that make it considerably more difficult to test than traditional software and hinder comprehensive coverage of potential failure scenarios. Through a systematic literature mapping of 247 relevant studies, this work constructs the first conceptual framework that maps robotic software testing onto established general software testing theory. It systematically categorizes, compares, and relates existing approaches, offering a structured overview of the current research landscape, core challenges, and representative practices. By identifying key open problems, the study provides a theoretical foundation and strategic guidance for advancing robotic software testing, thereby fostering interdisciplinary integration between software engineering and robotics and supporting the development of reliable, verifiable autonomous systems.

Technology Category

• Intelligent Robots: Other Foundations and Applications
• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Robotic systems are complex and safety-critical software systems. As such, they need to be tested thoroughly. Unfortunately, robot software is intrinsically hard to test compared to traditional software, mainly since the software needs to closely interact with hardware, account for uncertainty in its operational environment, handle disturbances, and act highly autonomously. However, given the large space in which robots operate, anticipating possible failures when designing tests is challenging. This paper presents a mapping study by considering robotics testing papers and relating them to the software testing theory. We consider 247 robotics testing papers and map them to software testing, discussing the state-of-the-art software testing in robotics with an illustrated example, and discuss current challenges. Forming the basis to introduce both the robotics and software engineering communities to software testing challenges. Finally, we identify open questions and lessons learned.

Problem

• robotics
• software testing
• autonomy
• safety-critical systems
• uncertainty

Innovation

• robotics software testing
• mapping study
• autonomous systems
• software engineering
• test challenges