Search papers, labs, and topics across Lattice.
This paper introduces a novel reinforcement learning benchmarking framework based on stochastic converse optimality, allowing for the generation of control-affine nonlinear systems with known optimal policies. The framework provides necessary and sufficient conditions for a value function and policy to be optimal, enabling the systematic creation of benchmark families through homotopy variations and randomized parameters. The authors validate their approach by automatically constructing diverse environments and evaluating standard RL methods against the ground-truth optimum, providing a reproducible foundation for RL benchmarking.
Finally, a rigorous RL benchmark: generate environments with *provably* optimal policies, enabling controlled algorithm evaluation against ground truth.
The objective comparison of Reinforcement Learning (RL) algorithms is notoriously complex as outcomes and benchmarking of performances of different RL approaches are critically sensitive to environmental design, reward structures, and stochasticity inherent in both algorithmic learning and environmental dynamics. To manage this complexity, we introduce a rigorous benchmarking framework by extending converse optimality to discrete-time, control-affine, nonlinear systems with noise. Our framework provides necessary and sufficient conditions, under which a prescribed value function and policy are optimal for constructed systems, enabling the systematic generation of benchmark families via homotopy variations and randomized parameters. We validate it by automatically constructing diverse environments, demonstrating our framework's capacity for a controlled and comprehensive evaluation across algorithms. By assessing standard methods against a ground-truth optimum, our work delivers a reproducible foundation for precise and rigorous RL benchmarking.
