EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation

2026-03-19

AI Summary

This work addresses the challenge of achieving both accuracy and efficiency with compact Vision Transformers (ViTs) on resource-constrained edge devices for dense prediction tasks. To this end, the authors propose EdgeCrafter, a framework that integrates task-specialized knowledge distillation, a lightweight ViT backbone, and an edge-friendly encoder-decoder architecture, significantly enhancing the performance of small-scale ViTs using only task-specific annotations. Experimental results demonstrate that ECDet-S achieves 51.7 AP on COCO object detection with fewer than 10M parameters; ECInsSeg matches the instance segmentation performance of RF-DETR with reduced model size; and ECPose-X attains 74.8 AP in pose estimation, substantially outperforming YOLO26Pose-X—which relies on large-scale pretraining—thereby validating the superior accuracy-parameter efficiency trade-off of the proposed approach.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Large Vision Models
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Cloud, edge and content delivery systems for the Web
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

Deploying high-performance dense prediction models on resource-constrained edge devices remains challenging due to strict limits on computation and memory. In practice, lightweight systems for object detection, instance segmentation, and pose estimation are still dominated by CNN-based architectures such as YOLO, while compact Vision Transformers (ViTs) often struggle to achieve similarly strong accuracy efficiency tradeoff, even with large scale pretraining. We argue that this gap is largely due to insufficient task specific representation learning in small scale ViTs, rather than an inherent mismatch between ViTs and edge dense prediction. To address this issue, we introduce EdgeCrafter, a unified compact ViT framework for edge dense prediction centered on ECDet, a detection model built from a distilled compact backbone and an edge-friendly encoder decoder design. On the COCO dataset, ECDet-S achieves 51.7 AP with fewer than 10M parameters using only COCO annotations. For instance segmentation, ECInsSeg achieves performance comparable to RF-DETR while using substantially fewer parameters. For pose estimation, ECPose-X reaches 74.8 AP, significantly outperforming YOLO26Pose-X (71.6 AP) despite the latter's reliance on extensive Objects365 pretraining. These results show that compact ViTs, when paired with task-specialized distillation and edge-aware design, can be a practical and competitive option for edge dense prediction. Code is available at: https://intellindust-ai-lab.github.io/projects/EdgeCrafter/

Problem

• dense prediction
• edge devices
• Vision Transformers
• model compression
• resource constraints

Innovation

• compact Vision Transformers
• task-specialized distillation
• edge dense prediction
• edge-friendly architecture
• knowledge distillation