HFGCN:Hypergraph Fusion Graph Convolutional Networks for Skeleton-Based Action Recognition

2025-01-19

AI Summary

Existing skeleton-based action recognition methods often neglect the biomechanical principles underlying human motion and employ overly coarse topological modeling. To address these limitations, this paper proposes HFGCN—a kinematics-driven multi-granularity hypergraph modeling framework. First, a body-part hierarchical topology is explicitly constructed based on geometric distances between joints and the torso, thereby encoding biomechanical constraints. Second, a hypergraph structure is built to capture high-order joint coordination patterns; it incorporates a temporal hypergraph attention module and a channel-wise hypergraph graph convolution module to jointly optimize topological representations. By integrating kinematic priors with multi-scale feature learning, HFGCN achieves state-of-the-art performance on three benchmark datasets—NTU RGB+D, NTU RGB+D 120, and NW-UCLA—demonstrating significant improvements in recognition accuracy.

Technology Category

• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: Video Understanding & Activity Analysis

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In recent years, action recognition has received much attention and wide application due to its important role in video understanding. Most of the researches on action recognition methods focused on improving the performance via various deep learning methods rather than the classification of skeleton points. The topological modeling between skeleton points and body parts was seldom considered. Although some studies have used a data-driven approach to classify the topology of the skeleton point, the nature of the skeleton point in terms of kinematics has not been taken into consideration. Therefore, in this paper, we draw on the theory of kinematics to adapt the topological relations of the skeleton point and propose a topological relation classification based on body parts and distance from core of body. To synthesize these topological relations for action recognition, we propose a novel Hypergraph Fusion Graph Convolutional Network (HFGCN). In particular, the proposed model is able to focus on the human skeleton points and the different body parts simultaneously, and thus construct the topology, which improves the recognition accuracy obviously. We use a hypergraph to represent the categorical relationships of these skeleton points and incorporate the hypergraph into a graph convolution network to model the higher-order relationships among the skeleton points and enhance the feature representation of the network. In addition, our proposed hypergraph attention module and hypergraph graph convolution module optimize topology modeling in temporal and channel dimensions, respectively, to further enhance the feature representation of the network. We conducted extensive experiments on three widely used datasets.The results validate that our proposed method can achieve the best performance when compared with the state-of-the-art skeleton-based methods.

Problem

• skeleton recognition
• action recognition
• accuracy improvement

Innovation

• Hypergraph Convolutional Network
• Skeleton-based Action Recognition
• Kinematic Theory Integration