Dynamic Co-Optimization Compiler: Leveraging Multi-Agent Reinforcement Learning for Enhanced DNN Accelerator Performance

2024-07-11

AI Summary

To address the challenge of simultaneously achieving high accuracy, low latency, and efficient optimization when deploying DNN models across heterogeneous hardware platforms, this paper proposes the Dynamic Collaborative Optimization Compiler (DCOC). DCOC introduces a novel three-agent multi-agent reinforcement learning (MARL) framework that jointly optimizes hardware configuration, operator scheduling, and memory mapping. It further incorporates a high-confidence-driven dynamic search space pruning mechanism, marking the first application of division-of-labor MARL to compiler-level hardware-software co-optimization for end-to-end automatic mapping. Evaluated on diverse DNN models, DCOC achieves up to 37.95% higher throughput and reduces optimization time by 42.2% compared to state-of-the-art compilation frameworks. This work significantly advances research on efficient, adaptive DNN deployment across hardware platforms.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This paper introduces a novel Dynamic Co-Optimization Compiler (DCOC), which employs an adaptive Multi-Agent Reinforcement Learning (MARL) framework to enhance the efficiency of mapping machine learning (ML) models, particularly Deep Neural Networks (DNNs), onto diverse hardware platforms. DCOC incorporates three specialized actor-critic agents within MARL, each dedicated to different optimization facets: one for hardware and two for software. This cooperative strategy results in an integrated hardware/software co-optimization approach, improving the precision and speed of DNN deployments. By focusing on high-confidence configurations, DCOC effectively reduces the search space, achieving remarkable performance over existing methods. Our results demonstrate that DCOC enhances throughput by up to 37.95% while reducing optimization time by up to 42.2% across various DNN models, outperforming current state-of-the-art frameworks.

Problem

• Enhances DNN accelerator performance
• Uses MARL for hardware/software co-optimization
• Reduces search space and optimization time

Innovation

• Uses Multi-Agent Reinforcement Learning
• Focuses on hardware/software co-optimization
• Reduces DNN deployment search space