CGTGait: Collaborative Graph and Transformer for Gait Emotion Recognition

2025-09-20

AI Summary

To address insufficient long-range temporal modeling in skeleton-based gait emotion recognition, this paper proposes a dual-stream architecture—Collaborative Graph Convolution and Transformer (CGT). The method integrates lightweight CGT modules with a bidirectional cross-stream fusion mechanism to jointly model joint-level spatial topology and global temporal dependencies, thereby enhancing discriminative spatiotemporal feature representation while reducing computational overhead. Evaluated on Emotion-Gait and ELMD datasets, CGT achieves state-of-the-art or competitive accuracy, with only 0.34G FLOPs inference cost—82.2% lower than baseline methods—significantly improving the efficiency–accuracy trade-off. The core contribution lies in the first effective integration of graph convolutional networks and Transformers for gait emotion recognition, and empirical validation that bidirectional cross-stream fusion is critical for complementary spatiotemporal feature learning.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Skeleton-based gait emotion recognition has received significant attention due to its wide-ranging applications. However, existing methods primarily focus on extracting spatial and local temporal motion information, failing to capture long-range temporal representations. In this paper, we propose extbf{CGTGait}, a novel framework that collaboratively integrates graph convolution and transformers to extract discriminative spatiotemporal features for gait emotion recognition. Specifically, CGTGait consists of multiple CGT blocks, where each block employs graph convolution to capture frame-level spatial topology and the transformer to model global temporal dependencies. Additionally, we introduce a Bidirectional Cross-Stream Fusion (BCSF) module to effectively aggregate posture and motion spatiotemporal features, facilitating the exchange of complementary information between the two streams. We evaluate our method on two widely used datasets, Emotion-Gait and ELMD, demonstrating that our CGTGait achieves state-of-the-art or at least competitive performance while reducing computational complexity by approximately extbf{82.2%} (only requiring 0.34G FLOPs) during testing. Code is available at small{https://github.com/githubzjj1/CGTGait.}

Problem

• Recognizing emotions from skeleton-based gait data
• Capturing long-range temporal dependencies in gait sequences
• Reducing computational complexity in gait emotion recognition

Innovation

• Graph convolution captures frame-level spatial topology
• Transformer models global temporal dependencies effectively
• Bidirectional module fuses posture and motion features