DinoAtten3D: Slice-Level Attention Aggregation of DinoV2 for 3D Brain MRI Anomaly Classification

2025-09-15

AI Summary

Addressing the challenges of label scarcity and severe class imbalance in 3D brain MRI abnormality classification, this paper proposes a slice-level attention aggregation framework built upon DINOv2. Methodologically, it leverages DINOv2 as a backbone to extract self-supervised 2D axial slice features and introduces a soft attention mechanism for weighted slice-level feature aggregation. A joint optimization objective comprising supervised contrastive loss and intra-class variance regularization is designed to enhance inter-class separability and intra-class feature consistency. Experiments on the ADNI dataset and an institutional headache cohort demonstrate that the method significantly improves classification performance for few-shot and long-tailed classes, effectively mitigating annotation dependency and distribution shift. Results validate its efficacy, robustness, and clinical applicability for anomaly detection in 3D medical imaging.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Adversarial Attacks & Robustness
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Anomaly detection and classification in medical imaging are critical for early diagnosis but remain challenging due to limited annotated data, class imbalance, and the high cost of expert labeling. Emerging vision foundation models such as DINOv2, pretrained on extensive, unlabeled datasets, offer generalized representations that can potentially alleviate these limitations. In this study, we propose an attention-based global aggregation framework tailored specifically for 3D medical image anomaly classification. Leveraging the self-supervised DINOv2 model as a pretrained feature extractor, our method processes individual 2D axial slices of brain MRIs, assigning adaptive slice-level importance weights through a soft attention mechanism. To further address data scarcity, we employ a composite loss function combining supervised contrastive learning with class-variance regularization, enhancing inter-class separability and intra-class consistency. We validate our framework on the ADNI dataset and an institutional multi-class headache cohort, demonstrating strong anomaly classification performance despite limited data availability and significant class imbalance. Our results highlight the efficacy of utilizing pretrained 2D foundation models combined with attention-based slice aggregation for robust volumetric anomaly detection in medical imaging. Our implementation is publicly available at https://github.com/Rafsani/DinoAtten3D.git.

Problem

• Classifying 3D brain MRI anomalies using attention-based slice aggregation
• Addressing data scarcity and class imbalance in medical imaging
• Leveraging pretrained DINOv2 for volumetric anomaly detection

Innovation

• Attention-based slice aggregation for 3D MRI
• DINOv2 pretrained feature extractor adaptation
• Composite loss with contrastive learning regularization