Search papers, labs, and topics across Lattice.
R2D-RL is a novel reinforcement learning environment that bridges the RoboCup 2D Soccer Simulation (RCSS2D) with modern Python-based multi-agent reinforcement learning (MARL) frameworks through shared-memory communication and cycle-level synchronization. This integration facilitates both full-field and scenario-based training, allowing for configurable opponents and advanced reward shaping techniques, such as expected possession value (EPV). The environment includes benchmarks for 11-vs-11 matches and demonstrates baseline results, highlighting its potential for enhancing MARL research in complex, dynamic settings.
R2D-RL transforms the RoboCup 2D Soccer landscape, enabling seamless integration of advanced MARL techniques with a robust soccer simulation environment.
Robot soccer is a challenging testbed for multi-agent reinforcement learning because it combines partial observability, cooperative and adversarial interaction, sparse rewards, and long-horizon tactical behavior. RoboCup 2D Soccer Simulation (RCSS2D) provides a mature robot-soccer platform, but its competition-oriented server-client architecture is difficult to use directly with modern Python-based MARL workflows. We introduce R2D-RL, a reinforcement learning environment that connects RCSS2D and HELIOS-based player clients to a Python MARL interface through shared-memory communication and cycle-level synchronization. R2D-RL supports full-field and scenario-based training with configurable opponents, Base discrete and Hybrid parameterized action spaces, action masks, expected possession value (EPV)-based reward shaping, and parallel execution. We provide front-goal scenarios and an 11-vs-11 full-field benchmark, together with baseline results.
