Improved Knowledge Distillation for Land-Use Image Classification

2026-06-12

AI Summary

This work proposes a knowledge distillation approach that integrates hard and soft supervision to reduce computational overhead in land use image classification while maintaining high accuracy. Using VGG16 as the teacher model and MobileNetV2 as the lightweight student model, the method jointly optimizes cross-entropy loss with ground-truth labels, KL divergence between teacher and student output distributions, and cosine similarity loss on intermediate feature representations. Evaluated on three land use datasets, the proposed framework achieves an average classification accuracy of 99.04%, significantly outperforming both the baseline student model and distillation strategies relying on a single loss component. The results demonstrate that the approach effectively enhances the performance of compact models while enabling efficient model compression.

Abstract

In the present article, an improved Knowledge Distillation (KD) framework has been proposed for efficient compression of deep convolutional neural networks for land-use image classification task. Motivated by the need to achieve competitive classification accuracy while reducing computational complexity, a teacher-student learning paradigm is adopted in which a VGG16 network transfers knowledge to a lightweight MobileNetV2 model. The proposed framework integrates hard supervision from ground truth labels with a soft supervision strategy that combines Kullback-Leibler divergence and Cosine Similarity losses. Experiments conducted on three land-use datasets show that the proposed KD-based method yields improved performance, and achieves an accuracy of 99.04%, outperforming both baseline student training and single-loss distillation approaches, while retaining substantial model compression.

Problem

• Knowledge Distillation
• Land-Use Image Classification
• Model Compression
• Deep Convolutional Neural Networks

Innovation

• Knowledge Distillation
• Land-Use Classification
• Model Compression
• Soft Supervision
• Cosine Similarity