Problem
Research questions and friction points this paper is trying to address.
Bitstream Action Recognition
Bitstream Corruption
Robustness
Byte Modeling
Bitstream Action Recognition is Byte Modeling
🤖 AI Summary
This study addresses action recognition failures caused by bitstream corruption and the dependency on pixel-level decoding. We propose BRACE, a dual-branch byte modeling framework that achieves corruption-resistant recognition in the representation space through repair-free anchor alignment and unreliable anchor suppression mechanisms. Furthermore, we construct a realistic bitstream corruption simulator and introduce BAR, the first large-scale benchmark dataset for this domain. Experimental results demonstrate that BRACE significantly outperforms existing methods in robustness under bitstream corruption scenarios. These findings effectively validate the superiority of byte-level modeling and alignment strategies, establishing a novel paradigm for understanding corrupted videos without relying on traditional pixel reconstruction.
📝 Abstract
Conventional action recognition typically relies on successful pixel decoding of the bitstream. However, bitstream corruption during storage or transmission may cause severe visual artifacts or even decoding failure, posing a significant challenge to reliable action recognition. Bitstream Action Recognition (BAR) aims to overcome the dependency on decoding and the vulnerability to corruption. In this paper, we propose a novel BAR framework, Bitstream Recognition via Anchoring Corrupted Embeddings (BRACE). BRACE is a dual-branch byte-modeling architecture that treats a corrupted bitstream and its intact counterpart as two byte realizations of the same action. This guides the generation of rich and stable representations for robustness to corruption through Intact-Anchored Representation Alignment (IARA). The intact representation serves as a stable anchor, and the corrupted one is aligned to it at the embedding and decision levels under Unreliable-Anchor Suppression (UAS), entirely in representation space and without repairing the bitstream. To address the scarcity of corrupted bitstreams in practice, we introduce the Real-world Bitstream Corruption Simulator (RBCS), a four-parameter simulator that reproduces bit-flip and byte-loss errors arising in transmission and storage. Building on RBCS, we construct the first large-scale BAR dataset (BAR-D), which comprises the BAR-Stanford40 and BAR-PPMI subsets and spans diverse corruption types and severity levels. Finally, we build a large benchmark on BAR-D involving 14 action recognition methods from the pixel, compressed, and bitstream domains. Extensive experiments demonstrate that BRACE has superior robustness to bitstream corruption than all comparison methods. Ablation studies further validate the effectiveness of the proposed RBCS augmentation and IARA.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Bitstream Action Recognition
Byte Modeling
Intact-Anchored Representation Alignment
Unreliable-Anchor Suppression
Real-world Bitstream Corruption Simulator
🔎 Similar Papers
No similar papers found.
