Search papers, labs, and topics across Lattice.
This study investigates the impact of different rhetorical patterns in AI-assisted information evaluation, moving beyond traditional directive rhetoric to explore how various styles can foster critical thinking and independent reasoning. By conducting a within-subject study with 98 participants on a fact verification task, the researchers found that Scaffold Explanation significantly improved accuracy and encouraged deeper reflection, while Alternative Framing was the most preferred style. The findings highlight the importance of rhetorical design in conversational agents, revealing trade-offs between user performance and satisfaction.
Scaffold Explanation boosts accuracy in AI-assisted evaluations, challenging the norm of directive rhetoric and enhancing user reflection.
Prior work on AI-assisted information evaluation has largely focused on what AI systems communicate, comparing explanation types and formats, with responses predominantly cast in directive rhetoric where the system delivers a verdict and the user passively accepts it. While debate-style interactions have recently shown promise in prompting critical evaluation over deference, the rhetorical patterns that structure AI responses and how they might induce reflection, uncertainty, or independent reasoning remain largely unexamined. To address this, we investigated eight rhetorical patterns known to induce contemplation: Intentional Misleading, Interpretive Alternative, Scaffold Explanation, Triggering Distrust, Information Distortion, Alternative Framing, Socratic Questioning, and an Oracle baseline. Through a within-subject study with n=98 participants on a hint-on-demand fact verification task, we observed preliminary evidence that Scaffold Explanation were associated with the highest accuracy gains, and encouraging deeper reflection. Surprisingly, the adversarial conditions also improved accuracy modestly. Participants preferred Alternative Framing most and Interpretive Alternative least, largely due to the latter's perceived time cost. We discuss the implications of designing conversational agents with varied rhetorical styles and the trade-offs among user performance, satisfaction, and contemplation.
