an interpretable vision transformer framework for automated brain tumor classification

2026-04-23

AI Summary

This study addresses the challenges of time-consuming and subjective manual diagnosis of brain tumors in MRI by proposing an interpretable four-class classification method based on Vision Transformer (ViT-B/16) to automatically distinguish glioma, meningioma, pituitary tumor, and healthy tissue. The approach incorporates clinically oriented preprocessing with CLAHE enhancement, two-stage fine-tuning, hybrid MixUp/CutMix augmentation, exponential moving average (EMA), and test-time augmentation (TTA) to significantly enhance model performance. Clinical interpretability is achieved through Attention Rollout–generated saliency maps. Evaluated on a dataset of 7,023 MRI scans, the model achieves 99.29% accuracy and a 99.25% macro F1-score, with perfect recall (100%) for both healthy tissue and meningioma, consistently outperforming CNN-based baselines.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Brain tumors represent one of the most critical neurological conditions, where early and accurate diagnosis is directly correlated with patient survival rates. Manual interpretation of Magnetic Resonance Imaging (MRI) scans is time-intensive, subject to inter-observer variability, and demands significant specialist expertise. This paper proposes a deep learning framework for automated four-class brain tumor classification distinguishing glioma, meningioma, pituitary tumor, and healthy brain tissue from a dataset of 7,023 MRI scans. The proposed system employs a Vision Transformer (ViT-B/16) pretrained on ImageNet-21k as the backbone, augmented with a clinically motivated preprocessing and training pipeline. Contrast Limited Adaptive Histogram Equalization (CLAHE) is applied to enhance local contrast and accentuate tumor boundaries invisible to standard normalization. A two-stage fine-tuning strategy is adopted: the classification head is warmed up with the backbone frozen, followed by full fine-tuning with discriminative learning rates. MixUp and CutMix augmentation is applied per batch to improve generalization. Exponential Moving Average (EMA) of weights and Test-Time Augmentation (TTA) further stabilize and boost performance. Attention Rollout visualization provides clinically interpretable heatmaps of the brain regions driving each prediction. The proposed model achieves a test accuracy of 99.29%, macro F1-score of 99.25%, and perfect recall on both healthy and meningioma classes, outperforming all CNN-based baselines

Problem

• brain tumor classification
• automated diagnosis
• MRI interpretation
• inter-observer variability
• early diagnosis

Innovation

• Vision Transformer
• Interpretable AI
• Brain Tumor Classification
• Two-stage Fine-tuning
• Attention Rollout