AIR: Complex Instruction Generation via Automatic Iterative Refinement

2025-02-25

AI Summary

To address the challenge of large language models (LLMs) unreliably following complex, multi-step, context-sensitive instructions with multiple constraints, this paper proposes the first document-driven, automated iterative refinement framework. Our method leverages web documents as authoritative sources to generate initial constrained instructions and introduces an LLM-as-judge mechanism that iteratively compares model outputs against source documents, dynamically injecting fidelity, structural coherence, and diversity constraints. We pioneer explicit constraint modeling and structured instruction synthesis techniques. We curate AIR-10K, a high-quality benchmark comprising 10,000 complex instructions derived from real-world documents. Extensive evaluation across multiple benchmarks demonstrates substantial improvements over state-of-the-art methods, significantly enhancing LLMs’ capability to faithfully execute intricate, multi-constraint, context-dependent instructions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

With the development of large language models, their ability to follow simple instructions has significantly improved. However, adhering to complex instructions remains a major challenge. Current approaches to generating complex instructions are often irrelevant to the current instruction requirements or suffer from limited scalability and diversity. Moreover, methods such as back-translation, while effective for simple instruction generation, fail to leverage the rich contents and structures in large web corpora. In this paper, we propose a novel automatic iterative refinement framework to generate complex instructions with constraints, which not only better reflects the requirements of real scenarios but also significantly enhances LLMs' ability to follow complex instructions. The AIR framework consists of two stages: (1)Generate an initial instruction from a document; (2)Iteratively refine instructions with LLM-as-judge guidance by comparing the model's output with the document to incorporate valuable constraints. Finally, we construct the AIR-10K dataset with 10K complex instructions and demonstrate that instructions generated with our approach significantly improve the model's ability to follow complex instructions, outperforming existing methods for instruction generation.

Problem

• Generates complex instruction with constraints
• Improves LLMs' ability to follow instructions
• Enhances scalability and diversity of instructions

Innovation

• Automatic iterative refinement framework
• LLM-as-judge guidance
• AIR-10K dataset construction