Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

2024-07-01

AI Summary

To address the imbalance between generation quality and diversity in large language models (LLMs) under high-temperature sampling, this paper introduces *min-p* sampling—a novel token selection strategy that adaptively scales the probability threshold as *p*·max(*P*), dynamically discarding tokens whose probabilities fall below this threshold. Unlike conventional top-*p* (nucleus) sampling, which suffers from instability at elevated temperatures, *min-p* preserves textual coherence while enhancing creative diversity. Empirical evaluation on benchmarks including GPQA, GSM8K, and AlpacaEval demonstrates substantial improvements in both reasoning and open-ended generation performance. Human evaluations further confirm statistically significant gains in preference scores. The method has been adopted by multiple mainstream open-source LLM projects, establishing a new paradigm for controllable text generation that balances fidelity, fluency, and novelty without requiring architectural modifications or additional training.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Large Language Models (LLMs) generate text by sampling the next token from a probability distribution over the vocabulary at each decoding step. However, popular sampling methods like top-p (nucleus sampling) often struggle to balance quality and diversity, especially at higher temperatures, leading to incoherent or repetitive outputs. To address this challenge, we propose min-p sampling, a dynamic truncation method that adjusts the sampling threshold based on the model's confidence by scaling according to the top token's probability. We conduct extensive experiments on benchmarks including GPQA, GSM8K, and AlpacaEval Creative Writing, demonstrating that min-p sampling improves both the quality and diversity of generated text, particularly at high temperatures. Moreover, human evaluations reveal a clear preference for min-p sampling in terms of both text quality and diversity. Min-p sampling has been adopted by multiple open-source LLM implementations, highlighting its practical utility and potential impact.

Problem

• Balancing text quality and diversity in LLM outputs
• Addressing incoherent or repetitive outputs at high temperatures
• Improving creative and coherent text generation with min-p sampling

Innovation

• Min-p sampling adjusts threshold dynamically
• Improves text quality and diversity
• Adopted by multiple LLM implementations