PATS: Proficiency-Aware Temporal Sampling for Multi-View Sports Skill Assessment

2025-06-05

AI Summary

Existing video sampling methods disrupt temporal continuity, hindering the capture of discriminative motion patterns that distinguish experts from novices—leading to inaccurate automated assessment of motor skills. To address this, we propose a proficiency-aware adaptive temporal sampling strategy: (1) a novel skill-proficiency-guided dynamic time segmentation mechanism that preserves action-unit integrity while suppressing inter-segment redundancy; and (2) an integrated framework combining multi-view video analysis, action-completeness constrained optimization, and the SkillFormer architecture for unified modeling of highly dynamic, sequential movements. Evaluated on the EgoExo4D benchmark, our method achieves comprehensive improvements over state-of-the-art approaches (+0.65% to +3.05% in proficiency classification accuracy), with particularly notable gains in challenging domains—26.22% in rock climbing, 2.39% in music performance, and 1.13% in basketball—demonstrating significantly enhanced cross-task generalization capability.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Automated sports skill assessment requires capturing fundamental movement patterns that distinguish expert from novice performance, yet current video sampling methods disrupt the temporal continuity essential for proficiency evaluation. To this end, we introduce Proficiency-Aware Temporal Sampling (PATS), a novel sampling strategy that preserves complete fundamental movements within continuous temporal segments for multi-view skill assessment. PATS adaptively segments videos to ensure each analyzed portion contains full execution of critical performance components, repeating this process across multiple segments to maximize information coverage while maintaining temporal coherence. Evaluated on the EgoExo4D benchmark with SkillFormer, PATS surpasses the state-of-the-art accuracy across all viewing configurations (+0.65% to +3.05%) and delivers substantial gains in challenging domains (+26.22% bouldering, +2.39% music, +1.13% basketball). Systematic analysis reveals that PATS successfully adapts to diverse activity characteristics-from high-frequency sampling for dynamic sports to fine-grained segmentation for sequential skills-demonstrating its effectiveness as an adaptive approach to temporal sampling that advances automated skill assessment for real-world applications.

Problem

• Automated sports skill assessment lacks temporal continuity preservation
• Current video sampling disrupts critical movement pattern analysis
• Need adaptive temporal segmentation for multi-view proficiency evaluation

Innovation

• Proficiency-Aware Temporal Sampling preserves movement continuity
• Adaptive video segmentation ensures full execution capture
• Multi-view analysis boosts accuracy across diverse activities