Search papers, labs, and topics across Lattice.
PragAlign is a feedback-guided framework designed for controlled synthetic dialogue generation that ensures communicative intent, emotional expression, and natural flow by utilizing a generate-evaluate-revise loop. The framework employs an LLM-based evaluator that scores various aspects of dialogue quality and provides targeted feedback for refinement, achieving a remarkable 99.50% acceptance rate on 800 matched dialogue specifications. This approach highlights the importance of structured feedback in enhancing multi-constraint satisfaction, particularly in emotion alignment, while also revealing challenges in achieving consistent human-perceived quality in affective aspects of dialogue.
Achieving a staggering 99.50% acceptance rate in synthetic dialogue generation reveals the transformative power of feedback-guided refinement in meeting complex communicative constraints.
Synthetic dialogue generation can support research in privacy-restricted service settings, but generated conversations must preserve communicative intent, affective meaning, and natural dialogue flow. We introduce PragAlign, a feedback-guided framework for controlled synthetic dialogue generation conditioned on service context, target intent, and target emotion, with auxiliary trait-style controls. PragAlign uses a generate--evaluate--revise loop in which an LLM-based evaluator scores intent alignment, emotion alignment, coherence, fluency, and aggregate quality, then provides criterion-specific feedback for up to three refinement rounds. On 800 matched dialogue specifications, PragAlign achieves 99.50\% evaluator-defined acceptance, compared with 72.25\% for one-shot generation and 95.88\% for repeated generation without structured feedback. This indicates that repeated attempts account for much of the gain over one-shot generation, while structured feedback primarily improves last-mile multi-constraint satisfaction rather than broad average quality. Refinement gains are concentrated in emotion alignment, which is also the dominant failure mode in ablations. A separate human evaluation of 1,200 generated dialogues shows that intent expression and dialogue flow are highly recognizable to annotators, while emotion appropriateness is less stable and more subjective. These results support PragAlign as a quality-control framework for improving evaluator-defined communicative constraint satisfaction, while showing that affective realization and independent human-perceived quality remain open challenges.
