An Empirical Study on Embodied Artificial Intelligence Robot (EAIR) Software Bugs

2025-07-24

AI Summary

A lack of systematic understanding of failures in Embodied Artificial Intelligence Robot (EAIR) systems hinders robustness improvement. Method: We conduct an empirical study of 885 software defects from 80 open-source EAIR projects, employing manual annotation and iterative coding to identify 15 symptom categories, 18 root-cause categories, and 13 vulnerable modules. Contribution/Results: We introduce, for the first time, eight EAIR-specific symptom categories and eight EAIR-specific root-cause categories, and construct a symptom–cause–module mapping model. Our analysis reveals that the inherent complexity of AI agent reasoning and decision-making constitutes the primary source of EAIR-specific defects. This work establishes the first empirically grounded, EAIR-specific defect classification framework—providing foundational support for defect prediction, localization, and repair in embodied AI systems.

Technology Category

• Intelligent Robots: Embodied AI
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Embodied Artificial Intelligence Robots (EAIR) is an emerging and rapidly evolving technological domain. Ensuring their program correctness is fundamental to their successful deployment. However, a general and in-depth understanding of EAIR system bugs remains lacking, which hinders the development of practices and techniques to tackle EAIR system bugs. To bridge this gap, we conducted the first systematic study of 885 EAIR system bugs collected from 80 EAIR system projects to investigate their symptoms, underlying causes, and module distribution. Our analysis takes considerable effort, which classifies these bugs into 18 underlying causes, 15 distinct symptoms, and identifies 13 affected modules. It reveals several new interesting findings and implications which help shed light on future research on tackling or repairing EAIR system bugs. First, among the 15 identified symptoms, our findings highlight 8 symptoms specific to EAIR systems, which is characterized by severe functional failures and potential physical hazards. Second, within the 18 underlying causes, we define 8 EAIR-specific causes, the majority of which stem from the intricate issues of AI- agent reasoning and decision making. Finally, to facilitate precise and efficient bug prediction, detection, and repair, we constructed a mapping between underlying causes and the modules in which they most frequently occur, which enables researchers to focus diagnostic efforts on the modules most susceptible to specific bug types.

Problem

• Understanding symptoms and causes of EAIR system bugs
• Investigating module distribution of bugs in EAIR systems
• Mapping causes to modules for efficient bug detection

Innovation

• Systematic study of 885 EAIR system bugs
• Classified bugs into 18 causes and 15 symptoms
• Constructed mapping between causes and modules