Every Subtlety Counts: Fine-grained Person Independence Micro-Action Recognition via Distributionally Robust Optimization

2025-09-25

AI Summary

Micro-action recognition suffers from limited model generalization due to inter-subject variability. To address this, we propose a subject-agnostic distributionally robust learning framework that learns person-invariant representations. Our method introduces a time-frequency alignment module regularized by Wasserstein distance to disentangle subject-specific dynamics from spectral characteristics, and designs a group-invariant regularization loss integrating variance-guided perturbation, consistency-based fusion, and pseudo-group partitioning. A dual-branch network enables joint optimization at both feature and loss levels. Evaluated on the large-scale MA-52 benchmark, our approach significantly outperforms state-of-the-art methods in fine-grained micro-action recognition, achieving superior accuracy and cross-subject robustness—demonstrating strong and stable generalization capability.

Technology Category

• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Micro-action Recognition is vital for psychological assessment and human-computer interaction. However, existing methods often fail in real-world scenarios because inter-person variability causes the same action to manifest differently, hindering robust generalization. To address this, we propose the Person Independence Universal Micro-action Recognition Framework, which integrates Distributionally Robust Optimization principles to learn person-agnostic representations. Our framework contains two plug-and-play components operating at the feature and loss levels. At the feature level, the Temporal-Frequency Alignment Module normalizes person-specific motion characteristics with a dual-branch design: the temporal branch applies Wasserstein-regularized alignment to stabilize dynamic trajectories, while the frequency branch introduces variance-guided perturbations to enhance robustness against person-specific spectral differences. A consistency-driven fusion mechanism integrates both branches. At the loss level, the Group-Invariant Regularized Loss partitions samples into pseudo-groups to simulate unseen person-specific distributions. By up-weighting boundary cases and regularizing subgroup variance, it forces the model to generalize beyond easy or frequent samples, thus enhancing robustness to difficult variations. Experiments on the large-scale MA-52 dataset demonstrate that our framework outperforms existing methods in both accuracy and robustness, achieving stable generalization under fine-grained conditions.

Problem

• Recognizing subtle micro-actions despite inter-person variability differences
• Learning person-agnostic representations for robust generalization in real scenarios
• Addressing fine-grained motion variations through distributionally robust optimization

Innovation

• Distributionally Robust Optimization for person-agnostic representations
• Temporal-Frequency Alignment Module with dual-branch design
• Group-Invariant Regularized Loss with pseudo-group partitioning