Search papers, labs, and topics across Lattice.
This paper introduces Adversarial Heuristic Learning (AHL), a paradigm where AI agents refine game policies and supporting software by interpreting rules, selecting opponents, and analyzing replays, all while keeping model weights fixed. Evaluating this approach on the new AAArena benchmark, comprising 12 adversarial games and 1,920 human programs, reveals that Opus5.5 with Claude Code achieved top rankings in half of the games, demonstrating HL's potential but also highlighting challenges in complex rule understanding and long-horizon strategy. The study further shows that opponent selection and dense feedback are crucial for policy improvement, and agents can learn from both on-policy and off-policy replays.
AI agents can learn to climb leaderboards in complex adversarial games by refining executable policies, but struggle with intricate rule sets
Adversarial games have driven advances from heuristic search to reinforcement learning, yet learning and adapting strategies from limited samples remain challenging. AI agents offer an alternative by turning game experience into revisions of executable policies. Building on heuristic learning (HL), we formalize Adversarial Heuristic Learning (AHL), a paradigm that uses AI agents as learning engines to refine game policies and supporting software while keeping model weights fixed. We introduce AAArena, a benchmark comprising 12 authentic adversarial games and 1,920 archived human programs, with an evaluation protocol modeled on real-world game competitions. Agents interpret rules, choose opponents, analyze replays, and revise game agents to achieve their highest ranking within fixed match and evaluation budgets. We evaluate \val{completedmodels} model and harness configurations: Opus5.5 with Claude Code earns 6 gold medals, while no evaluated configuration tops the remaining 6 human ladders. Performance is generally weaker in games with more complex rule specifications. Further experiments show that opponent selection and dense feedback support policy improvement, and that agents learn from both on-policy replays of their own matches and off-policy replays of other players'matches. These results highlight HL's potential in adversarial games and identify persistent challenges in game understanding, strategy implementation, and long-horizon policy development.
Tsinghua AI