MATCHA: Efficient Deployment of Deep Neural Networks on Multi-Accelerator Heterogeneous Edge SoCs

2026-04-10

AI Summary

This work addresses the challenge of underutilized hardware when deploying deep neural networks (DNNs) on edge system-on-chips (SoCs) equipped with multiple heterogeneous accelerators. To overcome this limitation, the authors propose MATCHA, a unified deployment framework that, for the first time, integrates constraint programming with pattern matching to jointly optimize operator tiling, heterogeneous hardware mapping, and L3/L2 memory allocation and scheduling. This co-optimization enables highly concurrent execution across diverse accelerators, significantly improving resource utilization. Evaluated on the MLPerf Tiny benchmark suite, MATCHA reduces inference latency by up to 35% compared to the state-of-the-art MATCH compiler, demonstrating its effectiveness in enhancing DNN deployment efficiency on resource-constrained edge platforms.

Technology Category

• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Deploying DNNs on System-on-Chips (SoC) with multiple heterogeneous acceleration engines is challenging, and the majority of deployment frameworks cannot fully exploit heterogeneity. We present MATCHA, a unified DNN deployment framework that generates highly concurrent schedules for parallel, heterogeneous accelerators and uses constraint programming to optimize L3/L2 memory allocation and scheduling. Using pattern matching, tiling, and mapping across individual HW units enables parallel execution and high accelerator utilization. On the MLPerf Tiny benchmark, using a SoC with two heterogeneous accelerators, MATCHA improves accelerator utilization and reduces inference latency by up to 35% with respect to the the state-of-the-art MATCH compiler.

Problem

• DNN deployment
• heterogeneous accelerators
• multi-accelerator SoCs
• edge computing
• hardware heterogeneity

Innovation

• heterogeneous accelerators
• constraint programming
• memory allocation
• parallel scheduling
• edge SoC