Search papers, labs, and topics across Lattice.
This paper conducts the first comprehensive empirical evaluation of various fuzz testing methods for Reinforcement Learning (RL) agents, focusing on their effectiveness, diversity, efficiency, and practical utility across three increasingly complex environments. The study benchmarks five state-of-the-art fuzzing methods against random testing, revealing that throughput-oriented methods like MDPFuzz excel in crash discovery, while exploration-focused methods like SeqDivFuzz uncover a wider variety of crash behaviors. Importantly, the findings demonstrate that crashes identified through fuzz testing can significantly enhance agent robustness and facilitate effective safety monitoring, providing actionable insights for researchers and practitioners in the field.
Fuzz testing can dramatically improve RL agent robustness and safety monitoring, with specific methods outperforming others in crash discovery and diversity.
Reinforcement Learning (RL) agents are increasingly deployed in safety-critical domains such as robotics, autonomous driving, and drone control, where unexpected behaviors may lead to severe real-world consequences. Fuzz testing has recently emerged as a promising method for exploring the vast state spaces of RL agents and exposing crashes. Although numerous RL fuzzing methods have been proposed, existing studies often differ in evaluation settings, baselines, and metrics, making it difficult to draw reliable conclusions about their relative effectiveness and practical usefulness. To address this gap, we present the first comprehensive empirical study that systematically evaluates RL fuzzing methods from four complementary perspectives: effectiveness, diversity, efficiency, and practical utility. We benchmark five state-of-the-art methods alongside random testing under unified configurations across three environments of increasing complexity (MountainCar, BipedalWalker, and CARLA), and further assess the downstream usefulness of detected crashes for agent robustness improvement and safety monitoring. Our results reveal several key insights. For instance,throughput-oriented methods like MDPFuzz demonstrate superior effectiveness and efficiency in crash discovery, while methods explicitly designed to encourage exploration like SeqDivFuzz excel at uncovering diverse crash behaviors. We also show that fuzzing-generated crashes can meaningfully improve agent robustness and enable accurate safety monitoring with strong cross-method generalization. Beyond these empirical findings, we distill actionable guidance for both researchers and practitioners, highlighting the benefits of combining complementary fuzzing strategies and adopting multi-level diversity analysis to achieve more comprehensive and practical RL testing.
