efficient backbone design

To address the high computational overhead and memory bottlenecks hindering Vision Transformer (ViT) deployment on edge devices, this paper presents a systematic survey of lightweighting and acceleration techniques tailored for edge scenarios—spanning model compression (e.g., pruning, quantization, knowledge distillation, attention simplification), software optimization (e.g., compiler frameworks such as TVM), and hardware adaptation (e.g., GPU/TPU/FPGA mapping). Its key contributions include: (1) proposing the first unified taxonomy for ViT edge deployment, explicitly characterizing trade-offs among accuracy, latency, power consumption, and hardware platforms; (2) establishing a structured evaluation framework covering 120+ works to identify real-world deployment bottlenecks; and (3) delivering a reproducible, cross-platform technical selection guide to advance co-optimization of accuracy, latency, and power efficiency.

This work addresses the inefficiency of existing vision backbones on low-parallelism hardware such as CPUs, which are typically optimized for highly parallel accelerators. The authors propose design principles tailored for CPU deployment, emphasizing a balance between high multiply-accumulate operations per second (MACpS) and low latency, and introduce CPUBone—the first family of vision backbones explicitly optimized for CPUs. By incorporating grouped convolutions and small kernel sizes, CPUBone reduces computational load while enhancing execution efficiency on CPU hardware. Experiments demonstrate that CPUBone achieves state-of-the-art accuracy–speed trade-offs across diverse CPU platforms and exhibits strong transfer performance on downstream tasks including object detection and semantic segmentation.

This work addresses the challenge of achieving real-time performance in edge vision systems, where conventional multi-stage detection-classification pipelines suffer from fully GPU-serialized execution. The authors propose a five-step optimization methodology enabling zero-GPU-fallback INT8 deployment of classification models on NVIDIA Jetson Deep Learning Accelerators (DLAs), and construct a parallel inference pipeline with GPU-based detection and DLA-based classification. Key innovations include the first-ever DLA deployment workflow that entirely avoids GPU fallback, overcoming DLA operator limitations and quantization compatibility bottlenecks through techniques such as manual dynamic range calibration, quantization-aware training, and ONNX graph surgery. Evaluated on a Jetson Orin NX, the dual-head human attribute classifier operating in parallel with the detector incurs only a 0.8 FPS overhead (12.5 vs. 13.3 FPS) and supports cost-free scaling across dual DLAs.

This work addresses the lack of systematic methodologies in model optimization, which often relies on heuristic choices and struggles to accommodate diverse deployment constraints. It formalizes model compression and acceleration as a constraint-aware multi-objective engineering decision problem, establishing a unified and actionable framework grounded in five key dimensions: data availability, latency, memory footprint, accuracy tolerance, and retraining budget. By integrating techniques such as quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference optimization, the study proposes tailored optimization pipelines for four representative industrial scenarios, delivering a reproducible and quantifiable guide for technology selection.

This work addresses the significant discrepancy between traditional MACs-based efficiency metrics for vision backbones and actual inference latency on edge devices, which hinders hardware-efficient design. By analyzing the divergence between theoretical MACs and real-world execution times of common building blocks, the study identifies key factors governing hardware efficiency. It proposes LowFormer, a novel backbone featuring the lightweight Lowtention module as a replacement for multi-head self-attention. Through hardware-aware co-design of macro- and micro-architectures alongside cross-platform deployment optimizations, LowFormer achieves higher ImageNet accuracy while substantially outperforming state-of-the-art models in speed across diverse hardware platforms—including both edge and desktop GPUs—and demonstrates strong performance on downstream tasks such as classification, detection, and segmentation.

To address compilation optimization challenges for dynamically shaped models—particularly large language models (LLMs)—on heterogeneous backends, this paper proposes the first unified compilation abstraction enabling cross-level fusion across computational graphs, loop-level tensor programs, and external library calls. Our method introduces: (1) first-class symbolic shape annotations for global dynamic shape tracking, and (2) a multi-level intermediate representation (IR) fusion framework that integrates symbolic shape inference with shape-aware optimization scheduling. Evaluated on multiple GPU architectures, our approach achieves state-of-the-art performance. Moreover, it enables, for the first time, efficient end-to-end deployment of mainstream LLMs on resource-constrained platforms—including smartphones, embedded devices, and web browsers—without model retraining or structural modification. This significantly broadens the practical applicability of dynamically shaped models in real-world edge and client-side scenarios.

To address resource dynamics, multi-objective constraints (latency, utilization, privacy), and infrastructure instability in large language model (LLM) inference within heterogeneous edge environments, this paper proposes a runtime-reconfigurable framework for joint model partitioning and deployment optimization. It is the first to formulate layer-granular model partitioning and device placement as a dynamic constrained optimization problem, integrating model-aware capacity analysis, dynamic graph neural network–based repartitioning, and resource forecasting. Evaluated in a 6G multi-access edge computing scenario, the approach reduces end-to-end latency by 27.4% and improves average GPU utilization by 39.1% over static baselines, while enabling privacy-sensitive layers to execute locally. The core contribution lies in an online, constraint-adaptive inference scheduler that ensures theoretical rigor and practical deployability under time-varying operational conditions.

This work addresses the challenges of deploying Vision-Language-Action (VLA) models on edge robots, where real-time performance, cost, and energy constraints are critical, yet existing evaluations predominantly rely on desktop GPUs and overlook the potential of heterogeneous edge accelerators. The authors propose a model-hardware co-characterization methodology to construct the first cross-accelerator VLA performance benchmark, revealing a two-stage bottleneck: compute-intensive visual-language backbones and memory-intensive action experts. To mitigate these bottlenecks, they introduce DP-Cache and V-AEFusion optimization strategies that enable asynchronous pipelined parallelism. Experiments demonstrate speedups of 2.9× on GPUs and up to 6× on edge NPUs, with only marginal degradation in task success rates, thereby validating the feasibility of efficient, low-cost VLA deployment on resource-constrained edge platforms.

This study addresses the memory bottleneck in fine-tuning pretrained vision models on low-memory GPUs (e.g., 2 GB). The authors systematically evaluate five parameter-efficient fine-tuning (PEFT) methods and three gradient checkpointing strategies across diverse vision and vision-language models, assessing their accuracy, energy efficiency, and memory consumption. They propose a memory-budget-aware adaptive gradient checkpointing algorithm and extend the NetScore metric to incorporate deployment constraints. Experimental results demonstrate that QLoRA and BitFit reduce energy consumption by 20–30% with only a 1–2% accuracy drop; the adaptive checkpointing strategy lowers peak GPU memory usage by 43–79%; and self-supervised models such as DINOv2 surpass full fine-tuning performance on CIFAR-100 while consuming significantly fewer computational resources.

This work addresses the challenges of deploying large language models under constraints of memory, latency, and hardware cost, where existing post-training compression methods lack a unified and efficient solution for algorithm selection, precision allocation, and hardware adaptation. We propose an open-source, hardware-aware automated compression framework that enables end-to-end model compression with a single command. The framework features automatic model analysis, mixed-precision planning, and staged progressive quantization—from layers to blocks to the entire model. Innovatively, it establishes the first quantized checkpoint as a deployable baseline, ensuring all subsequent optimizations incrementally improve performance on the same model. This approach bridges algorithmic research and production deployment, significantly reducing resource overhead while preserving model accuracy, thereby enhancing the reproducibility and practicality of compression strategies.

Existing large-scale model training systems struggle to flexibly compose diverse parallelization strategies, often relying on manual expert tuning and lacking generality. This work proposes a programmable distributed training system that enables users to declaratively specify composite parallelism strategies—such as data, pipeline, and expert parallelism—through model annotations and scheduling directives. These specifications are compiled via a unified intermediate representation (IR) into device-level execution plans, fully decoupling strategy definition from runtime execution over a global compute-communication DAG. The system is the first to support automatic compilation of user-defined composite strategies, matching the performance of established approaches like ZeRO while significantly improving both performance and memory efficiency in complex scenarios such as DeepSeek-V3’s DualPipe.