Search papers, labs, and topics across Lattice.
This paper introduces DialogPII, a multilingual dataset designed to facilitate the detection of personally identifiable information (PII) in synthetic dialog transcripts across various domains. The dataset encompasses eight interaction scenarios and includes 19 entity types in 11 languages, generated through a combination of large language models and manual curation to ensure plausibility and contextual relevance. Key results demonstrate the dataset's utility in training and evaluating automatic de-identification systems, supported by baseline multilingual named entity recognition models and comprehensive validation metrics.
A multilingual dataset of synthetic dialogs reveals critical insights into personal information detection across diverse contexts and languages.
Conversational data collected in domains such as healthcare or social sciences is a valuable resource for research and automated analysis. However, responsible data sharing requires the detection and removal of personally identifiable and sensitive information to protect individual privacy. To support the development and evaluation of automatic de-identification systems, we present DialogPII, a multilingual dataset of synthetic dialogs and speech-derived transcripts for personal information detection. DialogPII covers eight interaction scenarios (emergency calls, medical anamnesis interviews, therapy sessions, insurance communication, customer support, clinical interviews regarding an AI-supported dashboard, police reports, and group therapy discussions), 19 entity types, and 11 languages (English, Arabic, Finnish, French, German, Hindi, Italian, Polish, Portuguese, Spanish, and Turkish). Dialogs were generated semi-automatically using large language models, manually curated for plausibility and diversity, and localized to country- and city-specific contexts. All dialogs were additionally converted to speech via text-to-speech synthesis, transcribed with Whisper, and annotated through automatic projection and manual correction, yielding aligned written and speech-derived resources across all languages. We further release baseline multilingual named entity recognition models and provide technical validation through inter-annotator agreement analysis, translation quality evaluation, annotation projection assessment, and benchmark experiments with transformer-based sequence labeling models.
