🤖 AI Summary
Existing 3D skeleton-based action recognition research suffers from fragmented representation taxonomies and evaluation protocols misaligned with real-world scenarios; moreover, mainstream datasets lack critical dimensions—including rear-view perspectives, multi-person interactions, fine-grained or violent actions, and pandemic-era behaviors. To address these gaps, we propose a four-dimensional taxonomy (dataset design, spatial modeling, temporal modeling, and signal enhancement) and introduce ANUBIS: the first large-scale, multi-view 3D skeleton dataset explicitly designed for realistic challenges. ANUBIS features rear-view captures, 101 action classes (including 21 pandemic-related behaviors), and standardized recordings from 128 participants using Azure Kinect’s multi-sensor fusion. We further establish a unified benchmark framework, enabling reproducible evaluation of 12 state-of-the-art models. Our analysis identifies temporal modeling capacity and signal robustness as the primary bottlenecks limiting current performance.
📝 Abstract
Skeleton-based action recognition, as a subarea of action recognition, is swiftly accumulating attention and popularity. The task is to recognize actions performed by human articulation points. Compared with other data modalities, 3D human skeleton representations have extensive unique desirable characteristics, including succinctness, robustness, racial-impartiality, and many more. We aim to provide a roadmap for new and existing researchers a on the landscapes of skeleton-based action recognition for new and existing researchers. To this end, we present a review in the form of a taxonomy on existing works of skeleton-based action recognition. We partition them into four major categories: (1) datasets; (2) extracting spatial features; (3) capturing temporal patterns; (4) improving signal quality. For each method, we provide concise yet informatively-sufficient descriptions. To promote more fair and comprehensive evaluation on existing approaches of skeleton-based action recognition, we collect ANUBIS, a large-scale human skeleton dataset. Compared with previously collected dataset, ANUBIS are advantageous in the following four aspects: (1) employing more recently released sensors; (2) containing novel back view; (3) encouraging high enthusiasm of subjects; (4) including actions of the COVID pandemic era. Using ANUBIS, we comparably benchmark performance of current skeleton-based action recognizers. At the end of this paper, we outlook future development of skeleton-based action recognition by listing several new technical problems. We believe they are valuable to solve in order to commercialize skeleton-based action recognition in the near future. The dataset of ANUBIS is available at: http://hcc-workshop.anu.edu.au/webs/anu101/home.
