Search papers, labs, and topics across Lattice.
This paper introduces the task of chess strategy verbalization, aiming to translate the complex strategies of chess engines into comprehensible natural language descriptions. The authors develop a pipeline for generating these descriptions and an evaluation framework to assess their quality objectively. Experiments reveal that natural language effectively conveys strategic insights, highlighting the need for evaluations that consider alternative strategies and the constraints of LLMs in this context.
Natural language can bridge the gap between superhuman chess engines and human players, making complex strategies more accessible and understandable.
Chess engines have long achieved superhuman playing strength. However, the underlying strategy behind their move suggestions is difficult for human players, even skilled ones, to comprehend. Motivated by this, we propose the task of chess strategy verbalization, which is to describe chess strategies in natural language. We design (i) a pipeline for verbalizing strategies and (ii) an evaluation framework for objective evaluation of generated strategy descriptions. Our experiments show that natural language is a promising and interpretable medium for communicating strategic information to both human and LLM players. We glean additional interesting insights, including (a) the importance of evaluating strategies beyond the main line, (b) the limitations of pure concept-based descriptions, and (c) the limitations of relying on LLMs rather than humans for evaluation.
