Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates how LLMs handle repair, a crucial aspect of human conversation, in multi-turn dialogues involving math problems. They analyze both model-initiated and user-initiated repair attempts, revealing significant differences in behavior across models like GPT and Claude. The study finds that LLMs exhibit distinct and often unreliable patterns in how they respond to repair cues, especially as conversations lengthen.
LLMs can be wildly inconsistent in multi-turn conversations, ranging from stubbornly ignoring corrections to being easily misled, revealing surprising vulnerabilities in their interactive behavior.
Repair, an important resource for resolving trouble in human-human conversation, remains underexplored in human-LLM interaction. In this study, we investigate how LLMs engage in the interactive process of repair in multi-turn dialogues around solvable and unsolvable math questions. We examine whether models initiate repair themselves and how they respond to user-initiated repair. Our results show strong differences across models: reactions range from being almost completely resistant to (appropriate) repair attempts to being highly susceptible and easily manipulated. We further demonstrate that once conversations extend beyond a single turn, model behavior becomes more distinctive and less predictable across systems. Overall, our findings indicate that each tested LLM exhibits its own characteristic form of unreliability in the context of repair.