UltraGen: Extremely Fine-grained Controllable Generation via Attribute Reconstruction and Global Preference Optimization

2025-02-17

AI Summary

Existing controllable text generation methods perform well under low-dimensional (3–5D) attribute control but suffer severe performance degradation in high-dimensional, fine-grained settings (e.g., 45D). This work proposes the first two-stage framework for ultra-fine-grained zero-shot controllable generation. Stage I introduces Attribute Reconstruction (AR), enabling soft/hard attribute fusion and efficient sampling-based correction to mitigate positional bias and attention dilution. Stage II employs Global Preference Optimization (GPO), jointly modeling massive soft and hard constraints via large-model-based soft attribute extraction, programmatically generated hard attributes, weakly supervised reconstruction, and Direct Preference Optimization (DPO). Experiments on 45-dimensional control demonstrate substantial improvements in Constraint Satisfaction Rate (CSR) and text quality, with fidelity significantly surpassing existing baselines.

Technology Category

• Natural Language Processing: Generation
• Knowledge Representation and Reasoning: Preferences
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Fine granularity is an essential requirement for controllable text generation, which has seen rapid growth with the ability of LLMs. However, existing methods focus mainly on a small set of attributes like 3 to 5, and their performance degrades significantly when the number of attributes increases to the next order of magnitude. To address this challenge, we propose a novel zero-shot approach for extremely fine-grained controllable generation (EFCG), proposing auto-reconstruction (AR) and global preference optimization (GPO). In the AR phase, we leverage LLMs to extract soft attributes (e.g., Emphasis on simplicity and minimalism in design) from raw texts, and combine them with programmatically derived hard attributes (e.g., The text should be between 300 and 400 words) to construct massive (around 45) multi-attribute requirements, which guide the fine-grained text reconstruction process under weak supervision. In the GPO phase, we apply direct preference optimization (DPO) to refine text generation under diverse attribute combinations, enabling efficient exploration of the global combination space. Additionally, we introduce an efficient attribute sampling strategy to identify and correct potentially erroneous attributes, further improving global optimization. Our framework significantly improves the constraint satisfaction rate (CSR) and text quality for EFCG by mitigating position bias and alleviating attention dilution.

Problem

• Enhance fine-grained text generation control
• Optimize global text attribute combinations
• Improve constraint satisfaction and text quality

Innovation

• Auto-reconstruction combines soft and hard attributes
• Global preference optimization refines text generation
• Efficient attribute sampling corrects erroneous attributes