Search papers, labs, and topics across Lattice.
This study investigates how language models (LLMs) perform in the game of Taboo, which requires players to describe a target word without using forbidden terms. By manipulating various constraints during the generative process, the authors assess the models' ability to balance compliance with lexical restrictions and communicative effectiveness, comparing their performance to that of human players. The findings reveal that while models struggle with lexical grounding under constraints, their strategies diverge from human players, highlighting significant limitations in LLMs' guessing capabilities and the complexities of effective communication in constrained environments.
LLMs struggle to balance rule compliance and effective communication in Taboo, revealing a significant gap in their guessing abilities compared to humans.
The game of Taboo requires describing a target word without using a set of forbidden words, so that other players can guess it. This deceptively simple task combines strict lexical constraints with the need for communicatively effective descriptions, making it a compelling playground for examining how LLMs navigate competing demands at inference time. We evaluate two open-weight models under conditions that intervene at progressively deeper levels of the generative process, from prompting to generation-time constraints to internal representations manipulations. We assess their outputs through forbidden word violation detection, LLM-as-a-judge measuring the degree to which generated descriptions successfully evoke the target concept for both human and machine guessers, and examining whether the strategies models adopt under constraint align with those of human players. Our results show that compliance with the rules of the game and communicative effectiveness trade off differently across conditions, and that models remain substantially weaker than humans as guessers, suggesting that lexical grounding under constraint is an open challenge for current language models.
