Optimizing ML Workload Partitioning between CPUs and CIM Accelerators for Heterogeneous Computing

2026-07-06

AI Summary

This work addresses the limitations of existing machine learning workload partitioning approaches for compute-in-memory (CIM) systems, which often overlook critical RRAM constraints—including storage capacity, high write latency, and endurance—and fail to exploit the full potential of CPU–CIM协同 computation. To overcome these challenges, the paper introduces the first unified integer linear programming (ILP) framework that jointly models RRAM physical constraints, parallelism, and heterogeneous resource scheduling to minimize end-to-end inference latency while respecting hardware limitations. By integrating empirical performance profiling with analytical modeling, the proposed method achieves globally optimal workload partitioning and enables design space exploration for CIM accelerators. Experimental results demonstrate significant speedups of 30.9× and 7.3× over CPU-only execution on edge and high-performance CPU platforms, respectively, substantially enhancing inference efficiency in heterogeneous systems.

Abstract

Computing-in-Memory (CIM) accelerators execute Matrix-Vector Multiplications (MVMs) in memory, making them a compelling solution for Machine Learning (ML) workloads. However, existing ML workload partitioning approaches for CIM accelerators do not fully account for Resistive Random Access Memory (RRAM) constraints such as limited memory, high write latency, and limited endurance. They also neglect parallelism, low-level architectural effects, or the Central Processing Unit (CPU) as a complementary compute resource. To address these limitations, we propose an Integer Linear Programming (ILP)-based workload partitioning framework for heterogeneous CPU-CIM systems. It minimizes end-to-end inference latency under RRAM constraints, captures parallelism, and combines empirical profiling with analytical models. Using our framework, heterogeneous CPU-CIM execution achieves speedups of up to 30.9x over CPU-only execution on an edge CPU and 7.3x over a high-performance CPU. A Design Space Exploration (DSE) yields further design insights for future CIM accelerators.

Problem

• Computing-in-Memory
• Machine Learning
• Workload Partitioning
• RRAM constraints
• Heterogeneous Computing

Innovation

• Computing-in-Memory
• workload partitioning
• Integer Linear Programming
• RRAM constraints
• heterogeneous computing