Search papers, labs, and topics across Lattice.
This paper explores Optimization under Input Uncertainty (OIU), highlighting the significance of incorporating realized input information to enhance optimization processes. By applying Rao-Blackwellization to reduce the variance of gradient estimators, the authors introduce Phenotype-Accelerated Evolutionary Strategy (PAES), which refines traditional Evolutionary Strategies (ES) for OIU scenarios. Experimental results demonstrate that PAES achieves faster convergence across a range of optimization tasks, including reinforcement learning benchmarks, compared to standard ES methods.
Leveraging realized input information can accelerate evolutionary optimization processes, achieving faster convergence in complex tasks.
We investigate Optimization under Input Uncertainty (OIU), in which the input to the objective function, rather than the objective function itself, is subject to uncertainty. OIU appears in manufacturing processes with production tolerance, control of physical systems with actuation noise, Mixture of Experts, and Reinforcement Learning (RL). Most of the existing approaches solve OIU by using the value of the objective function but discard the information of the realized input, even though the realized input is observable in various applications. The question here is whether the discarded information of the realized input is useful to accelerate the optimization process. We affirmatively answer this question for Evolutionary Strategy (ES) by theoretically showing that the information of the realized input can reduce the variance of the gradient estimator via Rao-Blackwellization. Using the Rao-Blackwellized gradient estimator, we propose Phenotype-Accelerated Evolutionary Strategy (PAES), which is a refinement of ES for OIU. Numerical experiments show that PAES converges faster than the usual ES from simple continuous optimization problems to RL benchmarks.
