MicroViTv2: Beyond the FLOPS for Edge Energy-Friendly Vision Transformers

2026-05-11

AI Summary

This work addresses the high computational cost and low energy efficiency of Vision Transformers on edge devices by proposing MicroViTv2, a hardware-aware lightweight visual Transformer. The key innovations include Single Depth-wise Transpose Attention (SDTA) for efficiently modeling long-range dependencies, along with reparameterized Patch Embedding (RepEmbed) and reparameterized Depth-wise Convolutional Mixer (RepDW) to accelerate inference. Evaluated on ImageNet-1K and COCO benchmarks, MicroViTv2 outperforms MobileViTv2, EdgeNeXt, and EfficientViT, achieving up to a 0.5% accuracy gain while delivering highly energy-efficient and low-latency inference on the Jetson AGX Orin platform.

Abstract

The Vision Transformer (ViT) achieves remarkable accuracy across visual tasks but remains computationally expensive for edge deployment. This paper presents MicroViTv2, a lightweight Vision Transformer optimized for real-device efficiency. Built upon the original MicroViT, the proposed model is designed based on reparameterized design, specifically Reparameterized Patch Embedding (RepEmbed) and Reparameterized Depth-Wise convolution mixer (RepDW) for faster inference, and introduces the Single Depth-Wise Transposed Attention (SDTA) to capture long-range dependencies with minimal redundancy. Despite slightly higher FLOPs, MicroViTv2 improves accuracy up to 0.5% compared to its predecessor and surpassing MobileViTv2, EdgeNeXt, and EfficientViT while maintaining fast inference and high energy efficiency on Jetson AGX Orin. Experiments on ImageNet-1K and COCO demonstrate that hardware-aware design and structural re-parameterization are key to achieving high accuracy and low energy consumption, validating the need to evaluate efficiency beyond FLOPs. Code is available at https://github.com/novendrastywn/MicroViT.

Problem

• Vision Transformer
• Edge Deployment
• Energy Efficiency
• Hardware-Aware Design
• Model Efficiency

Innovation

• Reparameterization
• Vision Transformer
• Edge Efficiency
• Depth-Wise Convolution
• Hardware-Aware Design