ExecuTorch -- A Unified PyTorch Solution to Run AI Models On-Device

2026-05-05

AI Summary

This work proposes ExecuTorch, the first end-to-end deployment framework natively integrated with the PyTorch ecosystem, addressing the fragmentation commonly encountered in edge AI deployment. By introducing an extensible backend abstraction, quantization-aware optimizations, and a unified model serialization format, ExecuTorch preserves the original model semantics while seamlessly targeting heterogeneous hardware—from microcontrollers to specialized accelerators—without sacrificing low latency or offline execution capabilities. The framework bridges the gap between research and production workflows, enabling consistent development and efficient deployment across a broad spectrum of devices, ranging from wearables to compute clusters, thereby significantly enhancing both deployment efficiency and cross-platform consistency.

Abstract

Local execution of AI on edge devices is important for low latency and offline operation. However, deploying models on diverse hardware remains fragmented, often requiring model conversion or complete reimplementation outside the PyTorch ecosystem where the model was originally authored. We introduce ExecuTorch, a unified PyTorch-native deployment framework for edge AI. ExecuTorch enables seamless deployment of machine learning models across heterogeneous compute environments. It scales from embedded microcontrollers to complex system-on-chips (SoCs) with dedicated accelerators, powering devices ranging from wearables and smartphones to large compute clusters. ExecuTorch preserves PyTorch semantics while allowing customization, support for optimizations like quantization, and pluggable execution "backends". These features together enable fast experimentation, allowing researchers to validate deployment behavior entirely within PyTorch, bridging the gap between research and production.

Problem

• edge AI
• model deployment
• hardware heterogeneity
• PyTorch ecosystem
• on-device execution

Innovation

• ExecuTorch
• on-device AI
• PyTorch-native deployment
• heterogeneous hardware
• pluggable backends