Boosting AI Reliability with an FSM-Driven Streaming Inference Pipeline: An Industrial Case

2026-03-02

AI Summary

Industrial AI systems often exhibit prediction bias and fragility when encountering scenarios not covered in the training data. To address this, this work proposes a streaming inference pipeline that integrates a finite state machine (FSM) with an object detection model, explicitly embedding the FSM into the streaming AI inference process for the first time. By leveraging prior knowledge of operational scenarios, the approach guides and corrects real-time predictions of excavator workload. The method injects domain knowledge in a structured manner through a rule-guided prediction correction mechanism, significantly enhancing the model’s logical consistency and robustness in unseen scenarios. Evaluated on a real-world dataset comprising 12 construction sites, over 7,000 images, and more than 300 excavation operations, the proposed method substantially outperforms baseline approaches relying on handcrafted heuristic rules.

Technology Category

• Intelligent Robots: State Estimation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The widespread adoption of AI in industry is often hampered by its limited robustness when faced with scenarios absent from training data, leading to prediction bias and vulnerabilities. To address this, we propose a novel streaming inference pipeline that enhances data-driven models by explicitly incorporating prior knowledge. This paper presents the work on an industrial AI application that automatically counts excavator workloads from surveillance videos. Our approach integrates an object detection model with a Finite State Machine (FSM), which encodes knowledge of operational scenarios to guide and correct the AI's predictions on streaming data. In experiments on a real-world dataset of over 7,000 images from 12 site videos, encompassing more than 300 excavator workloads, our method demonstrates superior performance and greater robustness compared to the original solution based on manual heuristic rules. We will release the code at https://github.com/thulab/video-streamling-inference-pipeline.

Problem

• AI robustness
• out-of-distribution scenarios
• prediction bias
• industrial AI reliability
• streaming inference

Innovation

• FSM-driven inference
• streaming inference pipeline
• AI robustness
• prior knowledge integration
• industrial AI application