UAMTERS: Uncertainty-Aware Mutation Analysis for DL-enabled Robotic Software

2026-02-23

AI Summary

Existing mutation analysis approaches struggle to effectively evaluate the testing capability of deep learning–driven robotic software under environmental uncertainty. To address this limitation, this work introduces uncertainty modeling into mutation analysis for the first time, proposing uncertainty-aware mutation operators tailored for deep learning–based robotic systems. These operators inject controllable stochastic uncertainty to simulate realistic behavioral deviations and are accompanied by novel mutation scoring metrics that quantify a test suite’s ability to detect failures across varying levels of uncertainty. Experimental evaluation on three robotic case studies demonstrates that the proposed method more accurately discriminates between test suite qualities and effectively captures software failures induced by environmental uncertainty.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Self-adaptive robots adjust their behaviors in response to unpredictable environmental changes. These robots often incorporate deep learning (DL) components into their software to support functionality such as perception, decision-making, and control, enhancing autonomy and self-adaptability. However, the inherent uncertainty of DL-enabled software makes it challenging to ensure its dependability in dynamic environments. Consequently, test generation techniques have been developed to test robot software, and classical mutation analysis injects faults into the software to assess the test suite's effectiveness in detecting the resulting failures. However, there is a lack of mutation analysis techniques to assess the effectiveness under the uncertainty inherent to DL-enabled software. To this end, we propose UAMTERS, an uncertainty-aware mutation analysis framework that introduces uncertainty-aware mutation operators to explicitly inject stochastic uncertainty into DL-enabled robotic software, simulating uncertainty in its behavior. We further propose mutation score metrics to quantify a test suite's ability to detect failures under varying levels of uncertainty. We evaluate UAMTERS across three robotic case studies, demonstrating that UAMTERS more effectively distinguishes test suite quality and captures uncertainty-induced failures in DL-enabled software.

Problem

• uncertainty
• mutation analysis
• deep learning
• robotic software
• dependability

Innovation

• Uncertainty-Aware Mutation Analysis
• DL-enabled Robotic Software
• Stochastic Uncertainty Injection
• Mutation Score Metrics
• Self-adaptive Robots