Problem
Research questions and friction points this paper is trying to address.
Partition-Aware Scheduling
Heterogeneous Platforms
Inference Latency
DAG of Operators
Mobile Inference
Partition-Aware Scheduling for Mobile Heterogeneous Inference Co-Execution
🤖 AI Summary
该研究针对移动异构推理任务，提出一种分区感知调度方法，通过在线迭代搜索框架优化DAG调度，实现低延迟和高效执行。
📝 Abstract
Modern mobile inference runs on heterogeneous platforms combining mobile GPUs with multiple CPU core clusters. Existing optimizations typically exploit either inter-operator parallelism, by assigning entire operators to CPU cores or to the GPU, or intra-operator parallelism, by partitioning each operator for CPU-GPU co-execution. We consider these two forms of parallelism together, to improve inference latency of tasks that can be represented by a static DAG of operators with predefined input/output tensor shapes (e.g., CNNs or vision transformers). We define the problem of partition-aware DAG scheduling for mobile heterogeneous inference, illustrating that the best strategy depends on the structure of the inference DAG, thus motivating a joint formulation capturing operator partition choices, device assignment, and execution order. We propose an online iterative search framework, which decomposes large DAGs into stages, focuses search on critical operators, and uses latency predictors to estimate partitioned execution without exhaustive profiling. Across representative mobile inference workloads, our approach achieves latency close to an offline solution while keeping scheduling overhead to a fraction of the model initialization cost, allowing platform-specific scheduling at deployment time.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Partition-Aware Scheduling
Heterogeneous Inference
Online Iterative Search
Latency Optimization
🔎 Similar Papers
No similar papers found.
