Search papers, labs, and topics across Lattice.
The paper introduces ReaPER$++$, an annealed replay buffer rule for deep RL in quantum circuit optimization, transitioning from TD error-driven prioritization to reliability-aware sampling as training progresses, leading to significant sample efficiency gains. To address the quantum-classical evaluation bottleneck, they propose OptCRLQAS, which amortizes expensive evaluations over multiple architectural edits, reducing wall-clock time. Finally, they introduce a replay-buffer transfer scheme that reuses noiseless trajectories to warm-start noisy-setting learning, significantly reducing steps to chemical accuracy.
Forget painstakingly tuning RL algorithms for quantum circuit optimization – smart replay buffer engineering alone can slash training time by up to 90% and boost sample efficiency by 32x.
Deep reinforcement learning (RL) for quantum circuit optimization faces three fundamental bottlenecks: replay buffers that ignore the reliability of temporal-difference (TD) targets, curriculum-based architecture search that triggers a full quantum-classical evaluation at every environment step, and the routine discard of noiseless trajectories when retraining under hardware noise. We address all three by treating the replay buffer as a primary algorithmic lever for quantum optimization. We introduce ReaPER$+$, an annealed replay rule that transitions from TD error-driven prioritization early in training to reliability-aware sampling as value estimates mature, achieving $4-32\times$ gains in sample efficiency over fixed PER, ReaPER, and uniform replay while consistently discovering more compact circuits across quantum compilation and QAS benchmarks; validation on LunarLander-v3 confirms the principle is domain-agnostic. Furthermore we eliminate the quantum-classical evaluation bottleneck in curriculum RL by introducing OptCRLQAS which amortizes expensive evaluations over multiple architectural edits, cutting wall-clock time per episode by up to $67.5\%$ on a 12-qubit optimization problem without degrading solution quality. Finally we introduce a lightweight replay-buffer transfer scheme that warm-starts noisy-setting learning by reusing noiseless trajectories, without network-weight transfer or $\epsilon$-greedy pretraining. This reduces steps to chemical accuracy by up to $85-90\%$ and final energy error by up to $90\%$ over from-scratch baselines on 6-, 8-, and 12-qubit molecular tasks. Together, these results establish that experience storage, sampling, and transfer are decisive levers for scalable, noise-robust quantum circuit optimization.
