A3C-S: Automated Agent Accelerator Co-Search towards Efficient Deep Reinforcement Learning

2021-06-11

Design Automation Conference

AI Summary

To address the computational resource constraints and high energy consumption hindering deep reinforcement learning (DRL) deployment on edge devices (e.g., smartphones, smartwatches), this paper proposes the first agent–hardware accelerator co-automated search framework. Unlike conventional decoupled optimization, our approach jointly models DRL agent neural architecture search (NAS) and hardware-aware accelerator design as a multi-objective optimization problem. We introduce three key innovations: (1) an RL-based meta-controller for hierarchical policy learning, (2) a differentiable hardware surrogate model for accurate latency and energy estimation, and (3) multi-objective Bayesian optimization for Pareto-optimal trade-off discovery. Evaluated across multiple DRL benchmarks, our method achieves an average 12.7% improvement in test score, 38.5% reduction in energy consumption, and 41.2% decrease in inference latency over state-of-the-art methods, while demonstrating strong cross-platform generalization.

Abstract

Driven by the explosive interest in applying deep reinforcement learning (DRL) agents to numerous real-time control and decision-making applications, there has been a growing demand to deploy DRL agents to empower daily-life intelligent devices, while the prohibitive complexity of DRL stands at odds with limited on-device resources. In this work, we propose an Automated Agent Accelerator Co-Search (A3C-S) framework, which to our best knowledge is the first to automatically co-search the optimally matched DRL agents and accelerators that maximize both test scores and hardware efficiency. Extensive experiments consistently validate the superiority of our A3C-S over state-of-the-art techniques.

Problem

• Deep Reinforcement Learning
• Energy Efficiency
• Mobile Devices

Innovation

• A3C-S
• DRL Optimization
• Resource-constrained Devices