FOCUS: Fine-grained Optimization with Semantic Guided Understanding for Pedestrian Attributes Recognition

2025-06-28

AI Summary

In pedestrian attribute recognition (PAR), existing region-based feature methods suffer from loss of attribute-specific fine-grained patterns and poor generalization to unseen attributes. To address these limitations, we propose a semantic-guided adaptive attribute-level feature learning framework comprising three core components: (i) Multi-Granularity Mixed Tokens (MGMT) for cross-scale discriminative representation modeling; (ii) Attribute-Guided Visual Feature Extraction (AVFE) for fine-grained, attribute-specific feature learning; and (iii) Region-Aware Contrastive Learning (RACL) to enhance image–text alignment and interpretability. Our approach is the first to systematically support unified recognition of both seen and unseen attributes in PAR without requiring attribute label fine-tuning. Extensive experiments on PA100K, PETA, and RAPv1 demonstrate significant improvements in fine-grained accuracy and zero-shot generalization capability, achieving state-of-the-art performance and validating the effectiveness of the proposed framework.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Pedestrian attribute recognition (PAR) is a fundamental perception task in intelligent transportation and security. To tackle this fine-grained task, most existing methods focus on extracting regional features to enrich attribute information. However, a regional feature is typically used to predict a fixed set of pre-defined attributes in these methods, which limits the performance and practicality in two aspects: 1) Regional features may compromise fine-grained patterns unique to certain attributes in favor of capturing common characteristics shared across attributes. 2) Regional features cannot generalize to predict unseen attributes in the test time. In this paper, we propose the extbf{F}ine-grained extbf{O}ptimization with semanti extbf{C} g extbf{U}ided under extbf{S}tanding (FOCUS) approach for PAR, which adaptively extracts fine-grained attribute-level features for each attribute individually, regardless of whether the attributes are seen or not during training. Specifically, we propose the Multi-Granularity Mix Tokens (MGMT) to capture latent features at varying levels of visual granularity, thereby enriching the diversity of the extracted information. Next, we introduce the Attribute-guided Visual Feature Extraction (AVFE) module, which leverages textual attributes as queries to retrieve their corresponding visual attribute features from the Mix Tokens using a cross-attention mechanism. To ensure that textual attributes focus on the appropriate Mix Tokens, we further incorporate a Region-Aware Contrastive Learning (RACL) method, encouraging attributes within the same region to share consistent attention maps. Extensive experiments on PA100K, PETA, and RAPv1 datasets demonstrate the effectiveness and strong generalization ability of our method.

Problem

• Improves fine-grained attribute recognition in pedestrian images
• Addresses limitations of regional features in predicting unseen attributes
• Enhances generalization with adaptive attribute-level feature extraction

Innovation

• Adaptive fine-grained attribute-level feature extraction
• Multi-Granularity Mix Tokens for diverse visual features
• Region-Aware Contrastive Learning for consistent attention