Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait

2025-05-07

AI Summary

This paper addresses the challenging problem of unconstrained full-body person identification under high-altitude, long-range imaging conditions with severe interference—including large pose and scale variations, image degradation, and cross-domain discrepancies. To this end, we propose FarSight, an end-to-end system featuring three key innovations: (1) a novel quality-guided multimodal fusion mechanism; (2) recognition-oriented modules for video restoration, multi-person tracking, and modality-specific feature encoding; and (3) a dynamic weight fusion network coupled with quality-aware confidence modeling. FarSight jointly leverages facial, body shape, and gait features to enhance robustness under degradation. On the BRIAR benchmark, it achieves +34.1% TAR@0.1% FAR in 1:1 verification, +17.8% Rank-20 accuracy in closed-set identification, and −34.3% FNIR@1% FPIR in open-set identification. The system has also been validated through the NIST 2025 Face Identification in Video Evaluation (FIVE).

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Large-scale security measurements

Abstract

We address the problem of whole-body person recognition in unconstrained environments. This problem arises in surveillance scenarios such as those in the IARPA Biometric Recognition and Identification at Altitude and Range (BRIAR) program, where biometric data is captured at long standoff distances, elevated viewing angles, and under adverse atmospheric conditions (e.g., turbulence and high wind velocity). To this end, we propose FarSight, a unified end-to-end system for person recognition that integrates complementary biometric cues across face, gait, and body shape modalities. FarSight incorporates novel algorithms across four core modules: multi-subject detection and tracking, recognition-aware video restoration, modality-specific biometric feature encoding, and quality-guided multi-modal fusion. These components are designed to work cohesively under degraded image conditions, large pose and scale variations, and cross-domain gaps. Extensive experiments on the BRIAR dataset, one of the most comprehensive benchmarks for long-range, multi-modal biometric recognition, demonstrate the effectiveness of FarSight. Compared to our preliminary system, this system achieves a 34.1% absolute gain in 1:1 verification accuracy (TAR@0.1% FAR), a 17.8% increase in closed-set identification (Rank-20), and a 34.3% reduction in open-set identification errors (FNIR@1% FPIR). Furthermore, FarSight was evaluated in the 2025 NIST RTE Face in Video Evaluation (FIVE), which conducts standardized face recognition testing on the BRIAR dataset. These results establish FarSight as a state-of-the-art solution for operational biometric recognition in challenging real-world conditions.

Problem

• Recognizing persons at long distances and high angles
• Integrating face, gait, and body shape for identification
• Overcoming degraded image conditions and pose variations

Innovation

• Fusion of face, gait, and body shape modalities
• End-to-end system with four novel core modules
• Quality-guided multi-modal fusion under degraded conditions