A VLM-based Method for Visual Anomaly Detection in Robotic Scientific Laboratories

2025-06-04

AI Summary

Real-time detection of visual anomalies in robotic science laboratory experimental procedures remains challenging. Method: This paper proposes a Vision-Language Model (VLM)-driven multi-level prompting inference framework enabling progressive anomaly identification—from unsupervised to strongly supervised regimes. Contribution/Results: We introduce a novel four-tiered progressive prompting configuration framework and establish the first benchmark for visual anomaly detection in scientific experimental workflows, featuring first-person-view videos with fine-grained spatiotemporal annotations; we empirically validate the critical role of first-person perspective in process-level anomaly recognition. Evaluations on two mainstream VLMs demonstrate that richer contextual information consistently improves detection accuracy, with substantial gains in real experimental step classification. To foster reproducibility and community advancement, we publicly release the benchmark dataset and a standardized, end-to-end evaluation framework tailored for laboratory-scenario anomaly detection.

Technology Category

• Computer Vision: Language and Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Agentic search

Abstract

In robot scientific laboratories, visual anomaly detection is important for the timely identification and resolution of potential faults or deviations. It has become a key factor in ensuring the stability and safety of experimental processes. To address this challenge, this paper proposes a VLM-based visual reasoning approach that supports different levels of supervision through four progressively informative prompt configurations. To systematically evaluate its effectiveness, we construct a visual benchmark tailored for process anomaly detection in scientific workflows. Experiments on two representative vision-language models show that detection accuracy improves as more contextual information is provided, confirming the effectiveness and adaptability of the proposed reasoning approach for process anomaly detection in scientific workflows. Furthermore, real-world validations at selected experimental steps confirm that first-person visual observation can effectively identify process-level anomalies. This work provides both a data-driven foundation and an evaluation framework for vision anomaly detection in scientific experiment workflows.

Problem

• Detecting visual anomalies in robotic scientific labs
• Improving accuracy with contextual prompt configurations
• Validating process-level anomaly detection in real-world steps

Innovation

• VLM-based visual reasoning for anomaly detection
• Four progressive prompt configurations for supervision
• Tailored benchmark for scientific workflow anomalies