Search papers, labs, and topics across Lattice.
This paper introduces a reinforcement learning framework that tackles the tactical complexities of curling, a game often likened to chess but underexplored in machine learning. By employing the Deep Deterministic Policy Gradient algorithm, the authors address key challenges such as continuous state and action spaces and stochastic outcomes influenced by player skill variability. The resulting agent not only matches the performance of a hand-crafted expert heuristic in a simplified version of the game but also provides a dense value estimate for tactical comparisons, paving the way for enhanced decision-making tools in curling.
Effective curling strategies can be learned entirely through self-supervision, matching expert heuristics without human-annotated data.
Curling is often referred to as "Chess on Ice", owing to the tactical complexity of its decision-making process. Yet unlike chess, curling remains largely underexplored from a machine learning perspective, with prior work confined mainly to statistical approaches. We propose a reinforcement learning framework capable of quantitatively evaluating and comparing tactical options in curling. The game poses several modeling challenges: continuous state and action spaces, stochastic action outcomes reflecting player skill variability, and state transitions that are highly sensitive to small perturbations in the executed action. To address them, we employ the Deep Deterministic Policy Gradient actor-critic algorithm, adapted to exploit the finite-horizon structure of the game. Our experiments show that effective curling strategies can be acquired in a fully self-supervised manner, without any human-annotated data: on a reduced four-rock variant, the learned agent matches a hand-crafted expert heuristic in a regime where that heuristic is close to optimal, a parity we quantify against the intrinsic hammer advantage of the variant. Beyond the resulting policy, the learned critic provides a dense value estimate over the entire continuous action space, enabling the quantitative comparison of tactical alternatives for applications such as post-game performance analysis and decision support during athlete preparation.
