JEDI: JEPA-to-Edge Distillation for Efficient Cropland Segmentation from Satellite Imagery

2026-09-07

AI Summary

本文提出JEDI框架，通过两阶段方法将大型视觉模型的知识蒸馏到轻量级模型中，以解决卫星图像农田分割中的效率问题。

Abstract

Large vision models provide useful representations for remote-sensing segmentation but are often too expensive for deployment at the satellite or field edge. Existing feature-level distillation methods also tend to assume similar teacher and student architectures and often stop feature alignment when task training begins. We introduce JEDI (JEPA-to-Edge Distillation), a two-stage framework that transfers representations from a large I-JEPA Vision Transformer teacher to a compact SegFormer student. First, JEDI aligns the student's terminal representation with the teacher's token space using cross-architecture projection and spatial alignment. It then jointly optimizes supervised segmentation, temperature-scaled response distillation, and persistent feature alignment throughout task adaptation. On CalCROP21, JEDI-B0 achieves 68.0 mean Intersection-over-Union (mIoU) with 4.04M parameters, improving over the standalone student by 16.0 points and coming within 2.0 points of the 70.0 mIoU achieved by the 639M-parameter teacher. We evaluate SegFormer B0, B1, and B2 students with 4.04M, 14.33M, and 28M parameters, respectively. Across all three variants, JEDI consistently outperforms response-, structure-, channel-, and relational-distillation baselines under the same teacher-student setting. These results show that persistent representation alignment is especially valuable under aggressive compression, substantially reducing model size and computation while preserving segmentation performance.

Problem

• Large Vision Models
• Remote Sensing Segmentation
• Feature-level Distillation
• Model Deployment
• Satellite Imagery

Innovation

• cross-architecture projection
• spatial alignment
• persistent feature alignment
• temperature-scaled response distillation