AgriKD: Cross-Architecture Knowledge Distillation for Efficient Leaf Disease Classification

2026-05-02

AI Summary

This work addresses the challenge of deploying computationally intensive Vision Transformers (ViTs) for leaf disease classification on resource-constrained edge devices by proposing AgriKD, a novel framework that achieves cross-architecture knowledge distillation from a ViT teacher to a lightweight CNN student—the first such approach in agricultural vision tasks. AgriKD employs a multi-objective distillation strategy that jointly leverages output logits, intermediate features, and relational structures to effectively bridge the representational gap between Transformers and CNNs while preserving global modeling capabilities. The resulting student model exhibits a 172× reduction in parameters, a 47.57× decrease in computational cost, and 18–22× lower inference latency, all while maintaining accuracy comparable to the teacher across multiple datasets. The model has been successfully deployed on Jetson and mobile platforms, enabling stable real-time inference.

Abstract

Automated leaf disease classification is critical for early disease detection in resource-constrained field environments. Vision Transformers (ViTs) provide strong representation capability by modeling long-range dependencies and inter-class relationships; however, their high computational cost makes them impractical for deployment on edge devices. As a result, existing approaches struggle to effectively transfer these rich representations to lightweight models. This paper introduces AgriKD, a cross-architecture knowledge distillation framework for efficient edge deployment, which transfers knowledge from a Vision Transformer (ViT) teacher to a compact convolutional student model. To bridge the representational gap between Transformer and CNN architectures, the proposed approach integrates multiple distillation objectives at the output, feature, and relational levels, where each objective captures a different aspect of the teacher knowledge. This enables the student model to better preserve and utilize transformer-derived global representations. Experiments on multiple leaf disease datasets show that the distilled student achieves performance comparable to the teacher while significantly improving efficiency, reducing model parameters by approximately 172 times, computational cost by 47.57 times, and inference latency by 18-22 times. Furthermore, the optimized model is deployed across multiple runtime formats, including ONNX, TFLite Float16, and TensorRT FP16, achieving consistent predictive performance with negligible accuracy degradation. Real-world deployment on NVIDIA Jetson edge devices and a mobile application demonstrates reliable real-time inference, highlighting the practicality of AgriKD for AI-powered agricultural applications in resource-constrained environments.

Problem

• leaf disease classification
• knowledge distillation
• Vision Transformers
• edge deployment
• cross-architecture

Innovation

• Knowledge Distillation
• Cross-Architecture
• Vision Transformer
• Edge Deployment
• Leaf Disease Classification