Uncertainty-Driven Action Quality Assessment

2022-07-29

arXiv.org

AI Summary

Existing action quality assessment (AQA) methods rely on deterministic score prediction, neglecting the inherent subjectivity and inter-annotator variability in expert judgments—leading to limited robustness and generalization. To address this, we propose the first probabilistic modeling framework explicitly accounting for scoring uncertainty: (1) we model inter-expert score discrepancies as a latent uncertainty distribution; (2) we introduce an uncertainty-weighted regression loss and an uncertainty-guided progressive curriculum learning strategy; and (3) we implement dynamic uncertainty-aware reweighting and score distribution modeling via a conditional variational autoencoder (CVAE). Evaluated on three major benchmarks—MTL-AQA, FineDiving, and JIGSAWS—our method achieves state-of-the-art performance, significantly improving resilience to annotator bias, sparse supervision, and cross-domain generalization.

Abstract

Automatic action quality assessment (AQA) has attracted increasing attention due to its wide applications. However, most existing AQA methods employ deterministic models to predict the final score for each action, while overlooking the subjectivity and diversity among expert judges during the scoring process. In this paper, we propose a novel probabilistic model, named Uncertainty-Driven AQA (UD-AQA), to utilize and capture the diversity among multiple judge scores. Specifically, we design a Conditional Variational Auto-Encoder (CVAE)-based module to encode the uncertainty in expert assessment, where multiple judge scores can be produced by sampling latent features from the learned latent space multiple times. To further utilize the uncertainty, we generate the estimation of uncertainty for each prediction, which is employed to re-weight AQA regression loss, effectively reducing the influence of uncertain samples during training. Moreover, we further design an uncertainty-guided training strategy to dynamically adjust the learning order of the samples from low uncertainty to high uncertainty. The experiments show that our proposed method achieves competitive results on three benchmarks including the Olympic events MTL-AQA and FineDiving, and the surgical skill JIGSAWS datasets.

Problem

• Action Quality Assessment
• Subjectivity in Scoring
• Diversity of Judgement

Innovation

• Uncertainty-driven AQA
• Conditional Variational Autoencoder (CVAE)
• Adaptive Training Process