CSTA: Spatial-Temporal Causal Adaptive Learning for Exemplar-Free Video Class-Incremental Learning

2025-01-13

AI Summary

This work addresses exemplar-free video class-incremental learning, targeting catastrophic forgetting induced by spatiotemporal feature coupling and preserving semantic consistency between appearance and motion. We propose a causality-driven adaptive adapter framework featuring a novel spatiotemporal causal distillation module and a counterfactual compensation mechanism—enabling causal modeling of spatial–temporal knowledge without exemplars to jointly support dynamic new-class adaptation and stable retention of old-class knowledge. The method integrates disentangled spatiotemporal adapters, contrastive knowledge preservation, and end-to-end differentiable optimization. Evaluated on mainstream video incremental learning benchmarks, our approach achieves a new state-of-the-art average accuracy—surpassing existing exemplar-based methods by 4.2%—while requiring zero historical sample storage.

Technology Category

• Machine Learning: Causal Learning
• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Continual learning aims to acquire new knowledge while retaining past information. Class-incremental learning (CIL) presents a challenging scenario where classes are introduced sequentially. For video data, the task becomes more complex than image data because it requires learning and preserving both spatial appearance and temporal action involvement. To address this challenge, we propose a novel exemplar-free framework that equips separate spatiotemporal adapters to learn new class patterns, accommodating the incremental information representation requirements unique to each class. While separate adapters are proven to mitigate forgetting and fit unique requirements, naively applying them hinders the intrinsic connection between spatial and temporal information increments, affecting the efficiency of representing newly learned class information. Motivated by this, we introduce two key innovations from a causal perspective. First, a causal distillation module is devised to maintain the relation between spatial-temporal knowledge for a more efficient representation. Second, a causal compensation mechanism is proposed to reduce the conflicts during increment and memorization between different types of information. Extensive experiments conducted on benchmark datasets demonstrate that our framework can achieve new state-of-the-art results, surpassing current example-based methods by 4.2% in accuracy on average.

Problem

• Incremental Learning
• Video Data
• Knowledge Retention

Innovation

• CSTA
• Causal Relation Preservation
• Contradiction Resolution Mechanism