Multi-objective hybrid knowledge distillation for efficient deep learning in smart agriculture

2025-12-23

AI Summary

To address the challenge of balancing computational efficiency and recognition accuracy for deep learning models on resource-constrained edge devices in smart agriculture, this paper proposes a lightweight yet high-accuracy CNN architecture coupled with a multi-objective hybrid knowledge distillation framework. The student network innovatively integrates inverted residual blocks with dense connections, while the framework jointly incorporates hard-label supervision, feature-level and response-level distillation, and self-distillation—marking the first application of such a combination in agricultural fine-grained recognition to enhance generalization. Using ResNet18 as the teacher model, our approach achieves 98.56% accuracy on rice seed variety identification—only 0.09% lower than the teacher—while reducing FLOPs by 2.7× and parameters by 10×. Moreover, it demonstrates robust high performance across four crop disease datasets.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Deploying deep learning models on resource-constrained edge devices remains a major challenge in smart agriculture due to the trade-off between computational efficiency and recognition accuracy. To address this challenge, this study proposes a hybrid knowledge distillation framework for developing a lightweight yet high-performance convolutional neural network. The proposed approach designs a customized student model that combines inverted residual blocks with dense connectivity and trains it under the guidance of a ResNet18 teacher network using a multi-objective strategy that integrates hard-label supervision, feature-level distillation, response-level distillation, and self-distillation. Experiments are conducted on a rice seed variety identification dataset containing nine varieties and further extended to four plant leaf disease datasets, including rice, potato, coffee, and corn, to evaluate generalization capability. On the rice seed variety classification task, the distilled student model achieves an accuracy of 98.56%, which is only 0.09% lower than the teacher model (98.65%), while requiring only 0.68 GFLOPs and approximately 1.07 million parameters. This corresponds to a reduction of about 2.7 times in computational cost and more than 10 times in model size compared with the ResNet18 teacher model. In addition, compared with representative pretrained models, the proposed student reduces the number of parameters by more than 6 times relative to DenseNet121 and by over 80 times compared with the Vision Transformer (ViT) architecture, while maintaining comparable or superior classification accuracy. Consistent performance gains across multiple plant leaf disease datasets further demonstrate the robustness, efficiency, and strong deployment potential of the proposed framework for hardware-limited smart agriculture systems.

Problem

• Develops a lightweight CNN for edge devices in agriculture
• Uses hybrid knowledge distillation to balance accuracy and efficiency
• Evaluates model on seed and leaf disease datasets for generalization

Innovation

• Hybrid knowledge distillation framework for lightweight CNN
• Multi-objective strategy integrates four distillation techniques
• Customized student model combines inverted residual and dense blocks