Putting People in LLMs' Shoes: Generating Better Answers via Question Rewriter

2024-08-20

arXiv.org

AI Summary

Ambiguous user queries degrade the quality of long-form question answering (LFQA) by large language models (LLMs), particularly when interacting with black-box LLMs. Method: This paper proposes an instance-level, single-turn query rewriter to enhance query interpretability for black-box LLMs. It introduces a novel direct preference optimization (DPO) training paradigm guided by an automated answer quality discriminator—eliminating the need for human annotations—and incorporates a lightweight, plug-and-play prompt optimization mechanism. Contribution/Results: Extensive experiments across multiple black-box LLMs and LFQA benchmarks demonstrate that the rewriter significantly improves answer quality, exhibits strong generalization across models and domains, and maintains practical deployability. The approach is computationally efficient and does not require model fine-tuning or API access beyond standard inference. All code is publicly released.

Technology Category

• Natural Language Processing: Question Answering
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) have demonstrated significant capabilities, particularly in the domain of question answering (QA). However, their effectiveness in QA is often undermined by the vagueness of user questions. To address this issue, we introduce single-round instance-level prompt optimization, referred to as question rewriter. By enhancing the intelligibility of human questions for black-box LLMs, our question rewriter improves the quality of generated answers. The rewriter is optimized using direct preference optimization based on feedback collected from automatic criteria for evaluating generated answers; therefore, its training does not require costly human annotations. The experiments across multiple black-box LLMs and long-form question answering (LFQA) datasets demonstrate the efficacy of our method. This paper provides a practical framework for training question rewriters and sets a precedent for future explorations in prompt optimization within LFQA tasks. Code is available at https://github.com/3244we/Question-Rewriter.

Problem

• Improves LLM answer quality
• Optimizes vague user questions
• Reduces need for human annotations

Innovation

• Single-round prompt optimization
• Feedback-based direct preference optimization
• Black-box LLMs question rewriter