KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification

2025-12-09

AI Summary

To address the need for real-time, lightweight classification of retinal OCT images into three age-related macular degeneration (AMD) categories—normal, drusen, and choroidal neovascularization (CNV)—this paper proposes KD-OCT, a novel knowledge distillation framework. It employs ConvNeXtV2-Large as the teacher and EfficientNet-B2 as the student, introducing the first OCT-specific joint soft- and hard-label distillation strategy. To further enhance student performance, we integrate teacher-side data augmentation, stochastic weight averaging (SWA), and focal loss optimization—collectively overcoming accuracy bottlenecks in fine-grained retinal disease classification with compact models. Evaluated on the Noor Eye Hospital OCT dataset, KD-OCT achieves 98.5% classification accuracy—nearly matching the teacher’s performance—while reducing model size by 7.3× and accelerating inference by 5.8×. It significantly outperforms existing OCT-based multi-scale and feature-fusion approaches. The implementation is publicly available.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide, with optical coherence tomography (OCT) serving as a cornerstone for early detection and management. However, deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands. Therefore, it is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment. In this study, a novel knowledge distillation framework, termed KD-OCT, is proposed to compress a high-performance ConvNeXtV2-Large teacher model, enhanced with advanced augmentations, stochastic weight averaging, and focal loss, into a lightweight EfficientNet-B2 student for classifying normal, drusen, and CNV cases. KD-OCT employs real-time distillation with a combined loss balancing soft teacher knowledge transfer and hard ground-truth supervision. The effectiveness of the proposed method is evaluated on the Noor Eye Hospital (NEH) dataset using patient-level cross-validation. Experimental results demonstrate that KD-OCT outperforms comparable multi-scale or feature-fusion OCT classifiers in efficiency- accuracy balance, achieving near-teacher performance with substantial reductions in model size and inference time. Despite the compression, the student model exceeds most existing frameworks, facilitating edge deployment for AMD screening. Code is available at https://github.com/erfan-nourbakhsh/KD- OCT.

Problem

• Compresses large retinal OCT models for clinical use
• Maintains high accuracy while reducing computational demands
• Enables real-time edge deployment for AMD screening

Innovation

• Knowledge distillation compresses large teacher into lightweight student
• Combined loss balances soft teacher knowledge and hard ground truth
• Achieves near-teacher performance with reduced model size and time