Lost in the Vibrations: Vision Language Models Fail the Dynamic Gauges Test

2026-04-19

AI Summary

Current vision-language models struggle to accurately interpret high-frequency pointer oscillations and trajectories in dynamic analog instruments, failing to meet metrology’s stringent requirements for traceability and reliability. This work introduces IEEE/ISO metrological standards into the evaluation framework for vision-language models, constructing a video dataset encompassing diverse instrument types and motion speeds. Leveraging video sequence analysis, zero-shot reasoning, uncertainty quantification, and multimodal fusion techniques, the study systematically evaluates state-of-the-art models such as GPT-5 and Gemini 3. Experimental results demonstrate that existing models cannot reliably interpret dynamic pointer behavior, lack the performance necessary to function as trustworthy synthetic instruments, and are therefore unsuitable for deployment in safety-critical industrial monitoring applications.

Technology Category

• Computer Vision: Language and Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The digital transformation of industrial manufacturing increasingly relies on the ability of autonomous robots to interact with legacy infrastructure, particularly analog gauges. While Vision-Language Models (VLMs) have demonstrated potential in zero-shot instrument recognition, their deployment in measurement systems remains constrained by an inherent inability to accurately analyze high-frequency temporal events and needle vibrations. This paper evaluates state-of-the-art models, including GPT-5 and Gemini 3, against the strict requirements of metrology and uncertainty quantification. To facilitate this evaluation, we introduce a novel dataset comprising video sequences of various gauge types: circular, linear, and Vernier, under diverse motion speed profiles. Our findings indicate that current VLMs exhibit limited ability in interpreting needle trajectories and scale semantics, failing to provide the traceability and reliability needed for safety-critical monitoring. The results demonstrate that these models have not yet achieved the performance necessary to be classified as trustworthy synthetic instruments under existing IEEE and ISO standards.

Problem

• Vision-Language Models
• analog gauges
• needle vibrations
• temporal dynamics
• metrology

Innovation

• Vision-Language Models
• dynamic gauges
• needle vibration
• uncertainty quantification
• synthetic instruments