Search papers, labs, and topics across Lattice.
This paper introduces Gimitest, a comprehensive framework designed to rigorously test reinforcement learning (RL) policies across diverse environments and conditions. By addressing the limitations of existing automated testing methods that focus on specific scenarios and algorithms, Gimitest enhances the reliability and safety of RL applications. The framework's effectiveness is demonstrated through its application to various RL policies within established environments like Farama Gymnasium and PettingZoo, showcasing its versatility and robustness.
Gimitest reveals that existing RL testing methods are insufficient, providing a robust framework that can significantly enhance policy reliability across diverse scenarios.
Reinforcement learning (RL) policies can be unsafe and vulnerable to attacks. Ensuring their reliability is often a pain point as existing automated testing methods target only selected environments, testing scenarios, and RL algorithms. To address this, we propose a comprehensive framework for testing single- and multi-agent RL policies under varying conditions. Our implementation of this framework, Gimitest, is an open-source tool that supports various gym frameworks and allows for modifications of their integrated components. This article describes the framework and details Gimitest's functionality and architecture. It showcases its effectiveness in testing multiple RL policies in environments such as the official Farama Gymnasium and PettingZoo.
