Search papers, labs, and topics across Lattice.
This paper introduces a framework for generating synthetic dialogue data for intent classification without relying on human-annotated seed data, using only intent definitions. By leveraging topic and style attributes, the framework enhances data diversity, while two novel post-hoc stylization models, Univ and Exam, transform LLM-generated utterances into varied, human-like styles. Experimental results indicate that the approach can achieve up to 93.3% of the performance of models trained on human-annotated data, highlighting that style diversity is more crucial than topic diversity in preventing spurious stylistic correlations.
Style diversity in synthetic data generation proves to be more critical than topic diversity, significantly boosting model performance without human annotations.
Generating high-utility synthetic data for intent classification typically requires human-annotated seed data, which is often unavailable in fast-paced industrial settings. In this paper, we propose a framework for synthetic dialogue generation that works entirely without human-annotated data, relying solely on intent definitions. Our proposed dialogue generation framework utilizes two different types of topic and style attributes to improve data diversity. Also, we propose two novel post-hoc stylization models called Univ and Exam to transform synthetic LLM-generated utterances into more varied, human-like linguistic styles. To enhance data quality, we utilize an LLM-as-a-judge filtering process. Experimental results on both industrial and public datasets demonstrate that the proposed approach achieves up to 93.3% of the performance obtained using human-annotated training data. Crucially, the findings reveal that style diversity is more critical than topic diversity for synthetic data utility, as it prevents models from learning spurious stylistic correlations. Furthermore, the study shows that incorporating style attributes during the generation process is more effective than post-hoc style adaptation.
