EX-FIQA: Leveraging Intermediate Early eXit Representations from Vision Transformers for Face Image Quality Assessment

2026-04-21

AI Summary

This work addresses the limitation of existing Vision Transformer (ViT)-based face image quality assessment methods that rely solely on final-layer features, thereby overlooking complementary quality cues present in intermediate layers. The authors systematically analyze the contribution of each ViT layer to quality prediction and propose a multi-exit score fusion framework that requires neither architectural modifications nor additional training. By integrating predictions from multiple layers through depth-wise weighted averaging, the method reveals— for the first time—that intermediate ViT layers contain valuable and complementary quality signals, challenging the prevailing assumption that only deep layers are informative. Extensive experiments across eight benchmark datasets and four face recognition models demonstrate that this approach consistently matches or exceeds the performance of single-exit baselines while significantly reducing computational overhead, thus achieving a flexible trade-off between accuracy and efficiency.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Face Image Quality Assessment is crucial for reliable face recognition systems, yet existing Vision Transformer-based approaches rely exclusively on final-layer representations, ignoring quality-relevant information captured at intermediate network depths. This paper presents the first comprehensive investigation of how intermediate representations within ViTs contribute to face quality assessment through early exit mechanisms and score fusion strategies. We systematically analyze all twelve transformer blocks of ViT-FIQA architectures, demonstrating that different depths capture distinct and complementary quality-relevant information, as evidenced by varying attention patterns and performance characteristics across network layers. We propose a score fusion framework that combines quality predictions from multiple transformer blocks without architectural modifications or additional training. Our early exit analysis reveals optimal performance-efficiency trade-offs, enabling significant computational savings while maintaining competitive performance. Through extensive evaluation across eight benchmark datasets using four FR models, we demonstrate that our fusion strategy improves upon single-exit approaches. Our proposed quality fusion approach employs depth-weighted averaging that assigns progressively higher importance to deeper transformer blocks, achieving the best quality assessment performance by effectively leveraging the hierarchical nature of feature learning in ViTs. Our work challenges the conventional wisdom that only deep features matter for face analysis, revealing that intermediate representations contain valuable information for quality assessment. The proposed framework offers practical benefits for real-world biometric systems by enabling adaptive computation based on resource constraints while maintaining competitive quality assessment capabilities.

Problem

• Face Image Quality Assessment
• Vision Transformers
• Intermediate Representations
• Early Exit
• Quality-relevant Information

Innovation

• Vision Transformers
• Face Image Quality Assessment
• Early Exit
• Intermediate Representations
• Score Fusion