M3GCLR: Multi-View Mini-Max Infinite Skeleton-Data Game Contrastive Learning For Skeleton-Based Action Recognition

2026-03-10

AI Summary

This work addresses key limitations in self-supervised skeleton-based action recognition—namely, inadequate modeling of view discrepancies, the absence of adversarial mechanisms, and uncontrolled augmentation perturbations—by proposing a game-theoretic contrastive learning framework. The approach formulates an infinite skeleton data game model with a corresponding equilibrium theorem and introduces a minimax adversarial optimization mechanism. To enhance discriminative feature learning, it incorporates multi-view mutual information, temporally averaged neutral anchors, and a dual-loss equilibrium optimizer. Extensive experiments demonstrate that the method achieves state-of-the-art or competitive performance on standard benchmarks, including NTU RGB+D 60/120 and PKU-MMD, with accuracies of 82.1% and 85.8% on the NTU-60 X-Sub and X-View splits, respectively.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In recent years, contrastive learning has drawn significant attention as an effective approach to reducing reliance on labeled data. However, existing methods for self-supervised skeleton-based action recognition still face three major limitations: insufficient modeling of view discrepancies, lack of effective adversarial mechanisms, and uncontrollable augmentation perturbations. To tackle these issues, we propose the Multi-view Mini-Max infinite skeleton-data Game Contrastive Learning for skeleton-based action Recognition (M3GCLR), a game-theoretic contrastive framework. First, we establish the Infinite Skeleton-data Game (ISG) model and the ISG equilibrium theorem, and further provide a rigorous proof, enabling mini-max optimization based on multi-view mutual information. Then, we generate normal-extreme data pairs through multi-view rotation augmentation and adopt temporally averaged input as a neutral anchor to achieve structural alignment, thereby explicitly characterizing perturbation strength. Next, leveraging the proposed equilibrium theorem, we construct a strongly adversarial mini-max skeleton-data game to encourage the model to mine richer action-discriminative information. Finally, we introduce the dual-loss equilibrium optimizer to optimize the game equilibrium, allowing the learning process to maximize action-relevant information while minimizing encoding redundancy, and we prove the equivalence between the proposed optimizer and the ISG model. Extensive Experiments show that M3GCLR achieves three-stream 82.1%, 85.8% accuracy on NTU RGB+D 60 (X-Sub, X-View) and 72.3%, 75.0% accuracy on NTU RGB+D 120 (X-Sub, X-Set). On PKU-MMD Part I and II, it attains 89.1%, 45.2% in three-stream respectively, all results matching or outperforming state-of-the-art performance. Ablation studies confirm the effectiveness of each component.

Problem

• skeleton-based action recognition
• contrastive learning
• view discrepancy
• adversarial mechanism
• augmentation perturbation

Innovation

• contrastive learning
• game-theoretic optimization
• skeleton-based action recognition
• multi-view augmentation
• adversarial training