Problem
Research questions and friction points this paper is trying to address.
Gait Recognition
Vocabulary-Guided
Vision-Language Models
Feature Extraction
Universal Gait Features
Vocabulary-Guided Gait Recognition
🤖 AI Summary
本文提出了一种词汇引导的步态识别方法Gait-World，通过视觉-语言模型理解步态概念，并设计了词汇关系映射器和步态细粒度检测器来解决模态异质性问题。
📝 Abstract
What is a gait? Appearance-based gait networks consider a gait as the human shape and motion information from images. Model-based gait networks treat a gait as the human inherent structure from points. However, the considerations remain vague for humans to comprehend truly. In this work, we introduce a novel paradigm Vocabulary-Guided Gait Recognition, dubbed Gait-World, which attempts to explore gait concepts through human vocabularies with Vision-Language Models (VLMs). Although VLMs have achieved the remarkable progress in various vision tasks, the cognitive capability regarding gait modalities remains limited. The success element in Gait-World is the proper vocabulary prompt where this paradigm carefully selects gait cycle actions as Vocabulary Base, bridging the gait and vocabulary feature spaces and further promoting human understanding for the gait. How to extract gait features? Although previous gait networks have made significant progress, learning solely from gait modalities on limited gait databases makes it difficult to learn universal gait features for practicality. Therefore, we propose the first Gait-World model, dubbed α-Gait, which guides the gait network learning with vocabulary knowledge from VLMs. However, due to the heterogeneity of the modalities, directly integrating vocabulary and gait features is highly challenging as they reside in different embedding spaces. To address the issues, α-Gait designs Vocabulary Relation Mapper and Gait Fine grained Detector to map and establish vocabulary relations in the gait space for detecting corresponding gait features. Extensive experiments on CASIA-B, CCPG, SUSTech1K, Gait3D and GREW reveal the potential value and research directions of vocabulary information from VLMs in the gait field.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Vocabulary-Guided Gait Recognition
Vision-Language Models (VLMs)
Vocabulary Relation Mapper
Gait Fine-grained Detector
\\u03b1-Gait
🔎 Similar Papers
No similar papers found.
