Search papers, labs, and topics across Lattice.
This paper adapts the AlphaZero algorithm to the asymmetric board game Tablut, where players have unequal piece counts and objectives. The key modification involves using separate policy and value heads for each player role to avoid conflicting evaluation functions. The authors address training instabilities like catastrophic forgetting by using C4 data augmentation, a larger replay buffer, and playing against past checkpoints, achieving a BayesElo rating of 1235.
AlphaZero can conquer asymmetric games, but only with architectural surgery and a hefty dose of stabilization techniques.
This work investigates the adaptation of the AlphaZero reinforcement learning algorithm to Tablut, an asymmetric historical board game featuring unequal piece counts and distinct player objectives (king capture versus king escape). While the original AlphaZero architecture successfully leverages a single policy and value head for symmetric games, applying it to asymmetric environments forces the network to learn two conflicting evaluation functions, which can hinder learning efficiency and performance. To address this, the core architecture is modified to use separate policy and value heads for each player role, while maintaining a shared residual trunk to learn common board features. During training, the asymmetric structure introduced training instabilities, notably catastrophic forgetting between the attacker and defender roles. These issues were mitigated by applying C4 data augmentation, increasing the replay buffer size, and having the model play 25 percent of training games against randomly sampled past checkpoints. Over 100 self-play iterations, the modified model demonstrated steady improvement, achieving a BayesElo rating of 1235 relative to a randomly initialized baseline. Training metrics also showed a significant decrease in policy entropy and average remaining pieces, reflecting increasingly focused and decisive play. Ultimately, the experiments confirm that AlphaZero's self-play framework can transfer to highly asymmetric games, provided that distinct policy/value heads and robust stabilization techniques are employed.
