🤖 AI Summary
Fine-grained visual categorization (FGVC) confronts dual challenges: difficulty in identifying discriminative local features and poor model interpretability. To address these, we propose Loupe—a lightweight, plug-and-play attention module that implicitly localizes discriminative regions without requiring part-level annotations, while simultaneously generating human-interpretable attention maps. Loupe integrates seamlessly with mainstream vision transformers (ViTs), including Swin Transformer, and is trained end-to-end with a composite loss function designed to jointly optimize classification accuracy and attention fidelity. On the CUB-200-2011 benchmark, integrating Loupe into Swin-Base boosts top-1 accuracy from 85.40% to 88.06% (+2.66%), outperforming strong baselines. Qualitative analysis confirms that Loupe’s attention maps precisely highlight semantically critical regions—e.g., bird beaks, wing patterns—thereby enhancing both predictive performance and decision transparency. This makes Loupe particularly suitable for high-stakes applications demanding reliability and interpretability, such as biodiversity monitoring and medical image diagnosis.
📝 Abstract
Fine-Grained Visual Classification (FGVC) is a critical and challenging area within computer vision, demanding the identification of highly subtle, localized visual cues. The importance of FGVC extends to critical applications such as biodiversity monitoring and medical diagnostics, where precision is paramount. While large-scale Vision Transformers have achieved state-of-the-art performance, their decision-making processes often lack the interpretability required for trust and verification in such domains. In this paper, we introduce The Loupe, a novel, lightweight, and plug-and-play attention module designed to be inserted into pre-trained backbones like the Swin Transformer. The Loupe is trained end-to-end with a composite loss function that implicitly guides the model to focus on the most discriminative object parts without requiring explicit part-level annotations. Our unique contribution lies in demonstrating that a simple, intrinsic attention mechanism can act as a powerful regularizer, significantly boosting performance while simultaneously providing clear visual explanations. Our experimental evaluation on the challenging CUB-200-2011 dataset shows that The Loupe improves the accuracy of a Swin-Base model from 85.40% to 88.06%, a significant gain of 2.66%. Crucially, our qualitative analysis of the learned attention maps reveals that The Loupe effectively localizes semantically meaningful features, providing a valuable tool for understanding and trusting the model's decision-making process.