World Model Failure Classification and Anomaly Detection for Autonomous Inspection

2026-02-17

AI Summary

This work addresses the challenge of unreliable sensor readings in industrial inspection robots caused by occlusions, limited viewpoints, or environmental anomalies, which hinder real-time task status assessment. The authors propose a hybrid framework that integrates supervised fault classification with unsupervised anomaly detection, uniquely combining conformal prediction and world models to enable policy-agnostic, distribution-free early discrimination among three states—success, known faults, and out-of-distribution anomalies—using compressed video inputs. The approach facilitates training data quality evaluation and model feedback, achieving over 90% recognition accuracy on both office and industrial instrument inspection datasets. It outperforms human observers in decision speed and has been successfully deployed on a Boston Dynamics Spot robot for real-time operation.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Autonomous inspection robots for monitoring industrial sites can reduce costs and risks associated with human-led inspection. However, accurate readings can be challenging due to occlusions, limited viewpoints, or unexpected environmental conditions. We propose a hybrid framework that combines supervised failure classification with anomaly detection, enabling classification of inspection tasks as a success, known failure, or anomaly (i.e., out-of-distribution) case. Our approach uses a world model backbone with compressed video inputs. This policy-agnostic, distribution-free framework determines classifications based on two decision functions set by conformal prediction (CP) thresholds before a human observer does. We evaluate the framework on gauge inspection feeds collected from office and industrial sites and demonstrate real-time deployment on a Boston Dynamics Spot. Experiments show over 90% accuracy in distinguishing between successes, failures, and OOD cases, with classifications occurring earlier than a human observer. These results highlight the potential for robust, anticipatory failure detection in autonomous inspection tasks or as a feedback signal for model training to assess and improve the quality of training data. Project website: https://autoinspection-classification.github.io

Problem

• World Model Failure
• Anomaly Detection
• Autonomous Inspection
• Out-of-Distribution
• Failure Classification

Innovation

• World Model
• Conformal Prediction
• Anomaly Detection
• Autonomous Inspection
• Failure Classification