Lighthouse RL: Sample-Efficient Circuit Optimization via Strategic Reset Points

2026-07-15

AI Summary

This work addresses the limitations of traditional analog circuit sizing methods, which suffer from poor generalization, and the low sampling efficiency of standard reinforcement learning in non-promising regions. To overcome these challenges, the authors propose a “lighthouse”-based strategic reset mechanism that initializes new episodes from high-performing configurations identified during training, thereby steering exploration toward effective regions of the design space. This approach significantly enhances sample efficiency, optimization success rate, and generalization capability, while functioning as a plug-and-play module compatible with any reinforcement learning framework. Experimental results on benchmark problems and two classes of analog circuits demonstrate up to a 1.72× improvement in sample efficiency, an increase in optimization success rate from 0–87% to 100%, and an improvement in out-of-distribution generalization success rate from 0–50% to 75%.

Abstract

In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.

Problem

• analog circuit sizing
• sample efficiency
• reinforcement learning
• black-box optimization
• generalization

Innovation

• sample-efficient reinforcement learning
• strategic reset
• lighthouse states
• analog circuit optimization
• black-box optimization