Can Masking Background and Object Reduce Static Bias for Zero-Shot Action Recognition?

2025-01-22

Conference on Multimedia Modeling

AI Summary

In zero-shot action recognition, models are prone to static biases—such as background and object cues—that distort action semantic modeling. To address this, we propose a dual-mask intervention mechanism: for the first time, learnable foreground/background mask modules explicitly decouple scene-redundant information from action semantics within a CLIP-based transfer framework, enabling unbiased vision–language alignment. Furthermore, we introduce contrastive action semantic distillation to enhance discriminative action representations. On UCF101 and HMDB51, our method achieves zero-shot accuracy improvements of +5.2% and +4.8%, respectively, while significantly reducing background confusion. This work establishes an interpretable and scalable paradigm for mitigating static biases in action recognition, advancing both robustness and generalizability of zero-shot models.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Problem

• Zero-shot Action Recognition
• Static Bias
• Accuracy Issues

Innovation

• Zero-shot Action Recognition
• Bias Mitigation
• Background and Object Occlusion