Robust Long-term Test-Time Adaptation for 3D Human Pose Estimation through Motion Discretization

2025-11-24

AI Summary

In online test-time adaptation (TTA) for 3D human pose estimation, self-supervised learning suffers from error accumulation and performance degradation due to reliance on imperfect predictions. To address this, we propose a long-horizon TTA framework grounded in motion discretization. Our approach comprises three key innovations: (1) mapping continuous motion sequences into a discrete latent space via unsupervised clustering to extract semantically stable anchor actions; (2) introducing an exponential moving average–driven soft reset mechanism that dynamically suppresses error propagation during model rollback; and (3) incorporating a self-replay strategy to enhance temporal consistency modeling for out-of-distribution video streams. Evaluated on extended test sequences, our method significantly outperforms existing TTA approaches, enabling robust and persistent modeling of subject-specific morphology and motion dynamics. It demonstrates both stability and consistent accuracy improvement over prolonged adaptation periods.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Life-Long and Continual Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Online test-time adaptation addresses the train-test domain gap by adapting the model on unlabeled streaming test inputs before making the final prediction. However, online adaptation for 3D human pose estimation suffers from error accumulation when relying on self-supervision with imperfect predictions, leading to degraded performance over time. To mitigate this fundamental challenge, we propose a novel solution that highlights the use of motion discretization. Specifically, we employ unsupervised clustering in the latent motion representation space to derive a set of anchor motions, whose regularity aids in supervising the human pose estimator and enables efficient self-replay. Additionally, we introduce an effective and efficient soft-reset mechanism by reverting the pose estimator to its exponential moving average during continuous adaptation. We examine long-term online adaptation by continuously adapting to out-of-domain streaming test videos of the same individual, which allows for the capture of consistent personal shape and motion traits throughout the streaming observation. By mitigating error accumulation, our solution enables robust exploitation of these personal traits for enhanced accuracy. Experiments demonstrate that our solution outperforms previous online test-time adaptation methods and validate our design choices.

Problem

• Mitigating error accumulation in 3D human pose estimation during online adaptation
• Addressing domain gaps through motion discretization and anchor motion supervision
• Enabling robust long-term adaptation to personal motion traits in videos

Innovation

• Motion discretization creates anchor motions for supervision
• Soft-reset mechanism reverts pose estimator to moving average
• Unsupervised clustering enables efficient self-replay during adaptation