Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs

2025-06-03

AI Summary

Traditional decoding methods (e.g., top-k, nucleus sampling) struggle to simultaneously ensure fluency, diversity, and coherence in text generation. To address this, we propose Adaptive Semantic-aware Typicality Sampling (ASTS), the first method to introduce a dynamic entropy-driven typicality modulation mechanism within the Locally Typical Sampling framework. ASTS jointly optimizes local typicality and global semantic consistency by integrating real-time entropy estimation, semantic alignment rewards, and repetition penalties. It employs a multi-objective weighted scoring function and a reinforcement learning–inspired reward-penalty adjustment module. Extensive experiments on story generation and summarization demonstrate significant improvements: 12.3% gain in MAUVE, 18.7% increase in diversity score, 9.5% reduction in perplexity, and markedly lower repetition rates—achieving superior generation quality without compromising efficiency.

Technology Category

• Natural Language Processing: Generation
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This chapter explores advancements in decoding strategies for large language models (LLMs), focusing on enhancing the Locally Typical Sampling (LTS) algorithm. Traditional decoding methods, such as top-k and nucleus sampling, often struggle to balance fluency, diversity, and coherence in text generation. To address these challenges, Adaptive Semantic-Aware Typicality Sampling (ASTS) is proposed as an improved version of LTS, incorporating dynamic entropy thresholding, multi-objective scoring, and reward-penalty adjustments. ASTS ensures contextually coherent and diverse text generation while maintaining computational efficiency. Its performance is evaluated across multiple benchmarks, including story generation and abstractive summarization, using metrics such as perplexity, MAUVE, and diversity scores. Experimental results demonstrate that ASTS outperforms existing sampling techniques by reducing repetition, enhancing semantic alignment, and improving fluency.

Problem

• Improving text generation balance in LLMs
• Enhancing Locally Typical Sampling algorithm
• Ensuring coherence and diversity efficiently

Innovation

• Adaptive Semantic-Aware Typicality Sampling (ASTS)
• Dynamic entropy thresholding for coherence
• Multi-objective scoring for diverse generation