Problem
Research questions and friction points this paper is trying to address.
text-based person anomaly retrieval
anomalous behavior
vision-language matching
cross-modal retrieval
pedestrian anomaly
Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval
🤖 AI Summary
This work addresses the challenge of fine-grained retrieval of anomalous pedestrian behaviors from large-scale image collections based on natural language descriptions. To tackle this problem, we propose a robust cross-modal retrieval framework that integrates heterogeneous vision-language embeddings through score alignment and iterative ensemble strategies to effectively fuse multi-model representations. Furthermore, we introduce a discrepancy-aware re-ranking mechanism to handle semantically ambiguous queries. The proposed approach significantly enhances the robustness and accuracy of cross-modal matching in complex scenarios, achieving state-of-the-art performance on the PAB benchmark with 90.92% mAP, 85.13% Recall@1, 97.72% Recall@5, and 98.68% Recall@10, thereby demonstrating its effectiveness.
📝 Abstract
Text-based person anomaly retrieval aims to retrieve pedestrians exhibiting anomalous behaviors from a large image gallery using natural language descriptions. Compared with conventional text-based person retrieval, this task requires fine-grained reasoning over pedestrian appearance, behaviors, object interactions, and scene context, making robust cross-modal matching significantly more challenging. This paper presents the GENAI4E team's solution to AI City Challenge 2026 Track 4. Our framework builds upon a strong retrieval backbone and progressively integrates heterogeneous vision-language embedding models through score alignment and iterative ensemble fusion, followed by disagreement-aware VLM reranking for ambiguous queries. On the official Pedestrian Anomaly Behavior (PAB) benchmark, our approach achieves 90.92% mAP, 85.13% Recall@1, 97.72% Recall@5, and 98.68% Recall@10, demonstrating the effectiveness of combining complementary vision-language representations with selective multimodal reasoning for large-scale text-based person anomaly retrieval.
Innovation
Methods, ideas, or system contributions that make the work stand out.
heterogeneous ensemble
disagreement-aware reranking
vision-language embedding
text-based person retrieval
anomaly behavior detection
🔎 Similar Papers
No similar papers found.
