ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency

2026-06-23

AI Summary

This work addresses the challenge of online error detection in procedural videos by proposing ESTANet, an efficient approach that requires neither additional annotations nor complex architectural designs. ESTANet innovatively leverages the prediction inconsistencies between standard and error-sensitive action detectors across varying temporal contexts, employing a lightweight multi-detector ensemble combined with a majority voting mechanism to achieve frame-level error identification. Notably, it is the first method to exploit the intrinsic predictive behavior of action detectors themselves, thereby circumventing the need for specialized model architectures or explicit supervisory signals. Experimental results demonstrate that ESTANet achieves state-of-the-art performance on the EgoPER, Assembly-101-O, and EPIC-Tent-O datasets while maintaining real-time inference efficiency.

Abstract

An efficient and accurate system for detecting errors in procedural tasks is crucial for supporting human needs in daily life, as it can provide instant notifications and guide people to correct mistakes. In this work, we study real-time online error detection in procedural videos from a simple but overlooked perspective: the prediction behavior of action detectors themselves. Instead of designing complex architectures or specialized supervision, we observe that action detectors naturally exhibit different prediction characteristics depending on their sensitivity to input dynamics and temporal context. We therefore propose ESTANet (Error-Sensitive and Temporally-vArying Network), a lightweight framework that detects errors by exploiting inconsistencies among action predictions produced by a small set of action detectors. We construct standard and error-sensitive action detectors that behave similarly on correct executions but respond differently when errors occur. Meanwhile, detectors operating with different temporal contexts further amplify prediction inconsistencies when the procedure deviates from the intended sequence. During inference, we detect errors by aggregating mismatches between standard and error-sensitive predictions through majority voting to flag frames that contain errors. Extensive experiments on EgoPER, Assembly-101-O, and EPIC-Tent-O demonstrate that ESTANet achieves state-of-the-art performance in online error detection while maintaining real-time efficiency with a lightweight architecture. Our results highlight that leveraging the intrinsic properties of action detectors can yield a powerful and practical solution for online error detection without increasing architectural design complexity.

Problem

• online error detection
• procedural videos
• action prediction inconsistency
• real-time error detection
• error-sensitive detection

Innovation

• prediction inconsistency
• online error detection
• action detectors
• temporal context
• lightweight framework