Search papers, labs, and topics across Lattice.
This paper introduces Lighthouse RL, a sample-efficient reinforcement learning method for optimizing analog circuit sizing by employing a strategic reset strategy that leverages high-performing configurations, termed "lighthouses." By initializing episodes from these lighthouses, the method effectively directs exploration toward more promising regions, significantly enhancing sample efficiency and optimization performance compared to traditional RL and Bayesian optimization techniques. Experimental results demonstrate improvements in sample efficiency (up to 1.72x faster), optimization success rates (100% vs. 0-87%), and generalization capabilities (75% vs. 0-50%) across benchmark problems and analog circuits.
Strategic reset points in Lighthouse RL can boost sample efficiency and optimization success rates, making it a game-changer for analog circuit design.
In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.
