Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the pragmatic reasoning abilities of large language models (LLMs) in multi-party collaborative tasks where participants have access to different information. By formalizing the concept of collaborative epistemic asymmetry, the authors assess how well LLMs adhere to Grice's cooperative principle as both speakers and listeners. The findings reveal that while LLMs can demonstrate pragmatic capabilities through specific prompting and post-training strategies, they struggle with effective communication when faced with incomplete information, leading to unrecognized failures in cooperation.
LLMs can exhibit pragmatic reasoning in collaborative tasks, but they often flout communication norms when information is asymmetrical, risking task success.
Fruitful collaborations rely on cooperative communications, including of contextual cues to incorporate into reasoning. The increasing use of LLMs in collaborative and agentic pipelines raises questions about the extent to which they exhibit these pragmatic capabilities, especially in scenarios where they may not have access to the same information as their collaborators. In this paper, we perform a novel investigation into the pragmatic reasoning capabilities of LLMs in a multi-party collaborative task under partial information conditions. We formalize a notion of collaborative epistemic asymmetry that explicitly connects objective task success to Grice's cooperative principle and empirically assess various LLMs' abilities to act cooperatively as both speakers and listeners, including both prompting and post-training strategies. Our results show that while LLMs exhibit certain pragmatic capabilities in collaborative settings, and these can be elicited through prompting and post-training, they still face challenges in pragmatic communication with incomplete information, and that certain failure modes do correlate with floutings of Grice's maxims that go unrecognized.