🤖 AI Summary
Why is constrained neural language generation still challenging? Although modern models produce fluent text, they often fail to strictly satisfy verifiable output requirements—such as factual consistency, safety compliance, and syntactic or format correctness—due to a persistent conceptual conflation of “conditions” with “constraints.” This work formally defines constraints as *output-verifiable, hard requirements*, distinguishing them from soft conditions or input prompts. We propose a unified taxonomy spanning task types, constraint granularity, and verification modalities. Through a systematic literature review and methodological critique, we identify and analyze four key research directions: decoding-time control, structured search, constraint injection, and trustworthy evaluation—while exposing critical gaps in current evaluation practices. Our framework establishes a rigorous theoretical foundation and methodological guidance for safe, controllable, and verifiably compliant language generation.
📝 Abstract
Recent advances in deep neural language models combined with the capacity of large scale datasets have accelerated the development of natural language generation systems that produce fluent and coherent texts (to various degrees of success) in a multitude of tasks and application contexts. However, controlling the output of these models for desired user and task needs is still an open challenge. This is crucial not only to customizing the content and style of the generated language, but also to their safe and reliable deployment in the real world. We present an extensive survey on the emerging topic of constrained neural language generation in which we formally define and categorize the problems of natural language generation by distinguishing between conditions and constraints (the latter being testable conditions on the output text instead of the input), present constrained text generation tasks, and review existing methods and evaluation metrics for constrained text generation. Our aim is to highlight recent progress and trends in this emerging field, informing on the most promising directions and limitations towards advancing the state-of-the-art of constrained neural language generation research.
