A Survey of Reinforcement Learning for Optimization in Automation

2024-08-28

2024 IEEE 20th International Conference on Automation Science and Engineering (CASE)

AI Summary

This paper systematically reviews reinforcement learning (RL) optimization in three core automation domains: manufacturing, energy systems, and robotics. It identifies five persistent challenges—low sample efficiency, insufficient safety and robustness, poor interpretability, limited transferability, and deployment difficulties—and, for the first time, comprehensively analyzes their root causes and interdependencies. To address these, the paper proposes an integrated methodological framework combining safety-constrained RL, interpretability-enhancing techniques, multi-agent RL (MARL), meta-learning, and policy-gradient methods. The study constructs a novel three-dimensional RL optimization taxonomy spanning methodologies, challenges, and application scenarios, incorporating state-of-the-art algorithms including DQN, PPO, and MARL. It provides authoritative literature references and practical implementation guidelines, delivering a structured knowledge framework and reusable solutions to bridge academic research and industrial deployment.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Machine Learning: Reinforcement Learning
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Reinforcement Learning (RL) has become a critical tool for optimization challenges within automation, leading to significant advancements in several areas. This review article examines the present landscape of RL within automation, with a particular focus on its roles in manufacturing, energy systems, and robotics. It delves into state-of-the-art methods, major challenges, and upcoming avenues of research within each sector, highlighting RL’s capacity to solve intricate optimization challenges. The paper reviews the advantages and constraints of RL-driven optimization methods in automation. It points out prevalent challenges encountered in RL optimization, including issues related to sample efficiency and scalability; safety and robustness; interpretability and trustworthiness; transfer learning and meta-learning; and real-world deployment and integration. It further explores prospective strategies and future research pathways to navigate these challenges. Additionally, the survey includes an comprehensive list of relevant research papers, making it an indispensable guide for scholars and practitioners keen on exploring this domain.

Problem

• Reinforcement Learning for optimization in automation
• Challenges in sample efficiency and scalability
• Future research pathways for RL in automation

Innovation

• Reinforcement Learning for optimization
• Focus on manufacturing, energy, robotics
• Addresses sample efficiency, scalability, safety