Multi-Agent Reinforcement Learning for Inverse Design in Photonic Integrated Circuits

2025-06-23

AI Summary

Photonic integrated circuit (PIC) inverse design is often limited by gradient-based optimization, which frequently converges to suboptimal local minima. Method: We propose a multi-agent reinforcement learning (MARL) framework that discretizes the continuous design space into a grid and represents structural geometry via binary variables; thousands of cooperative agents perform distributed decision-making without reliance on gradients, enabling scalable 2D/3D device modeling and sample-efficient, gradient-free policy training. Contribution/Results: Our approach significantly enhances global search capability and sample efficiency—achieving high-performance designs with only ~1,000 environment samples. Experiments across diverse photonic devices demonstrate consistent superiority over conventional gradient-based methods in both solution quality and robustness. The framework exhibits strong scalability and practical engineering applicability, offering a promising paradigm for gradient-free PIC inverse design.

Technology Category

• Search and Optimization: Learning to Search
• Multiagent Systems: Adversarial Agents
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Inverse design of photonic integrated circuits (PICs) has traditionally relied on gradientbased optimization. However, this approach is prone to end up in local minima, which results in suboptimal design functionality. As interest in PICs increases due to their potential for addressing modern hardware demands through optical computing, more adaptive optimization algorithms are needed. We present a reinforcement learning (RL) environment as well as multi-agent RL algorithms for the design of PICs. By discretizing the design space into a grid, we formulate the design task as an optimization problem with thousands of binary variables. We consider multiple two- and three-dimensional design tasks that represent PIC components for an optical computing system. By decomposing the design space into thousands of individual agents, our algorithms are able to optimize designs with only a few thousand environment samples. They outperform previous state-of-the-art gradient-based optimization in both twoand three-dimensional design tasks. Our work may also serve as a benchmark for further exploration of sample-efficient RL for inverse design in photonics.

Problem

• Overcoming local minima in gradient-based PIC optimization
• Developing adaptive multi-agent RL for photonic design
• Enhancing sample efficiency in inverse design tasks

Innovation

• Multi-agent RL optimizes photonic circuit design
• Discretized grid enables binary variable optimization
• Outperforms gradient-based methods in sample efficiency