Search papers, labs, and topics across Lattice.
The paper introduces DiscoTrace, a method for representing and comparing answering strategies by annotating answers with discourse acts and question interpretations using Rhetorical Structure Theory parses. Applying DiscoTrace to human answers from nine communities reveals diverse answering preferences, while LLMs lack this diversity and favor breadth over strategic depth. This discrepancy highlights a key difference in how humans and LLMs approach information-seeking question answering.
LLMs, unlike humans, fail to exhibit rhetorical diversity in question answering, even when explicitly prompted to mimic specific community guidelines.
We introduce DiscoTrace, a method to identify the rhetorical strategies that answerers use when responding to information-seeking questions. DiscoTrace represents answers as a sequence of question-related discourse acts paired with interpretations of the original question, annotated on top of rhetorical structure theory parses. Applying DiscoTrace to answers from nine different human communities reveals that communities have diverse preferences for answer construction. In contrast, LLMs do not exhibit rhetorical diversity in their answers, even when prompted to mimic specific human community answering guidelines. LLMs also systematically opt for breadth, addressing interpretations of questions that human answerers choose not to address. Our findings can guide the development of pragmatic LLM answerers that consider a range of strategies informed by context in QA.
