PragAlign: Feedback-Guided Pragmatic Alignment for Controlled Synthetic Dialogue Generation

2026-09-02

AI Summary

为解决合成对话生成中意图、情感和自然度的问题，PragAlign采用基于反馈的生成-评估-修订循环方法，显著提高对话质量。

Abstract

Synthetic dialogue generation can support research in privacy-restricted service settings, but generated conversations must preserve communicative intent, affective meaning, and natural dialogue flow. We introduce PragAlign, a feedback-guided framework for controlled synthetic dialogue generation conditioned on service context, target intent, and target emotion, with auxiliary trait-style controls. PragAlign uses a generate--evaluate--revise loop in which an LLM-based evaluator scores intent alignment, emotion alignment, coherence, fluency, and aggregate quality, then provides criterion-specific feedback for up to three refinement rounds. On 800 matched dialogue specifications, PragAlign achieves 99.50\% evaluator-defined acceptance, compared with 72.25\% for one-shot generation and 95.88\% for repeated generation without structured feedback. This indicates that repeated attempts account for much of the gain over one-shot generation, while structured feedback primarily improves last-mile multi-constraint satisfaction rather than broad average quality. Refinement gains are concentrated in emotion alignment, which is also the dominant failure mode in ablations. A separate human evaluation of 1,200 generated dialogues shows that intent expression and dialogue flow are highly recognizable to annotators, while emotion appropriateness is less stable and more subjective. These results support PragAlign as a quality-control framework for improving evaluator-defined communicative constraint satisfaction, while showing that affective realization and independent human-perceived quality remain open challenges.

Problem

• synthetic dialogue generation
• communicative intent
• affective meaning
• natural dialogue flow
• service context

Innovation

• Feedback-Guided
• Pragmatic Alignment
• Controlled Synthetic Dialogue Generation
• LLM-based Evaluator
• Multi-constraint Satisfaction