Semantic-guided Diverse Decoding for Large Language Model

2025-06-30

AI Summary

Current large language models (LLMs) achieve lexical-level diversity in decoding, yet fail to meet the semantic diversity requirements of tasks such as Best-of-N sampling, population-based reinforcement learning, and synthetic data generation. This work proposes an embedding-space-driven semantic diversity generation framework. It introduces three novel mechanisms: orthogonal direction guidance, dynamic inter-group repulsion, and position-debiased probability assessment—integrated with adaptive gain control and constrained optimization to enable controllable, semantic-level diversity during autoregressive decoding. Unlike conventional n-gram penalties or temperature scaling, our approach directly imposes structured semantic constraints in the latent space. Experiments demonstrate that the method improves Best-of-N semantic coverage by 1.4–5.2%, accelerates RLHF training convergence by 15%, and boosts downstream task accuracy by up to 2.1%.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Diverse decoding of large language models is crucial for applications requiring multiple semantically distinct responses, yet existing methods primarily achieve lexical rather than semantic diversity. This limitation significantly constrains Best-of-N strategies, group-based reinforcement learning, and data synthesis. While temperature sampling and diverse beam search modify token distributions or apply n-gram penalties, they fail to ensure meaningful semantic differentiation. We introduce Semantic-guided Diverse Decoding (SemDiD), operating directly in embedding space that balances quality with diversity through three complementary mechanisms: orthogonal directional guidance, dynamic inter-group repulsion, and position-debiased probability assessment. SemDiD harmonizes these competing objectives using adaptive gain functions and constraint optimization, ensuring both quality thresholds and maximal semantic differentiation. Experiments show SemDiD consistently outperforms existing methods, improving Best-of-N coverage by 1.4-5.2% across diverse tasks and accelerating RLHF training convergence by 15% while increasing accuracy by up to 2.1%.

Problem

• Achieving semantic diversity in large language model outputs
• Overcoming limitations of lexical diversity methods
• Balancing response quality with meaningful differentiation

Innovation

• Semantic-guided Diverse Decoding (SemDiD) in embedding space
• Orthogonal directional guidance for semantic differentiation
• Dynamic inter-group repulsion and position-debiased probability assessment