Search papers, labs, and topics across Lattice.
This paper investigates the privacy risks associated with using Large Language Models (LLMs) in Task-Oriented Dialogue Systems (TODS), focusing on the unintended memorization of training dialogue data. The authors demonstrate that existing training data extraction attacks are ineffective in the context of LLM-based TODS due to unique characteristics of dialogue modeling. They propose novel attack techniques tailored for LLM-based TODS, enhancing response sampling and membership inference, and achieve a best-case precision exceeding 70% in extracting thousands of dialogue state labels.
LLM-powered task bots leak training data like travel schedules with alarming precision, demanding new defenses.
Large Language Models (LLMs) have been widely adopted to enhance Task-Oriented Dialogue Systems (TODS) by modeling complex language patterns and delivering contextually appropriate responses. However, this integration introduces significant privacy risks, as LLMs, functioning as soft knowledge bases that compress extensive training data into rich knowledge representations, can inadvertently memorize training dialogue data containing not only identifiable information such as phone numbers but also entire dialogue-level events like complete travel schedules. Despite the critical nature of this privacy concern, how LLM memorization is inherited in developing task bots remains unexplored. In this work, we address this gap through a systematic quantitative study that involves evaluating existing training data extraction attacks, analyzing key characteristics of task-oriented dialogue modeling that render existing methods ineffective, and proposing novel attack techniques tailored for LLM-based TODS that enhance both response sampling and membership inference. Experimental results demonstrate the effectiveness of our proposed data extraction attack. Our method can extract thousands of training labels of dialogue states with best-case precision exceeding 70%. Furthermore, we provide an in-depth analysis of training data memorization in LLM-based TODS by identifying and quantifying key influencing factors and discussing targeted mitigation strategies.
