Entropy-Aligned Decoding of LMs for Better Writing and Reasoning

2026-01-05

arXiv.org

AI Summary

This work proposes EPIC, a novel decoding method that addresses the limitations of traditional language model decoding strategies, which often rely on greedy heuristics and produce homogeneous, repetitive, and incoherent text. EPIC is the first entropy-aligned decoding approach that operates without hyperparameters by dynamically incorporating the entropy of future trajectories into the decoding process, thereby aligning the sampling distribution with the inherent uncertainty of the data. It further introduces an entropy-aware Lazy Gumbel-Max sampling mechanism that ensures exactness while requiring only sublinear entropy computations. Experimental results demonstrate that EPIC significantly outperforms existing decoding strategies across creative writing, summarization, and mathematical reasoning tasks, consistently improving LM-as-judge preference win rates while simultaneously enhancing output diversity and summary faithfulness.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generations. Decoding algorithms attempt to restrict the LM distribution to a set of high-probability continuations, but rely on greedy heuristics that introduce myopic distortions, yielding sentences that are homogeneous, repetitive and incoherent. In this paper, we introduce EPIC, a hyperparameter-free decoding approach that incorporates the entropy of future trajectories into LM decoding. EPIC explicitly regulates the amount of uncertainty expressed at every step of generation, aligning the sampling distribution's entropy to the aleatoric (data) uncertainty. Through Entropy-Aware Lazy Gumbel-Max sampling, EPIC manages to be exact, while also being efficient, requiring only a sublinear number of entropy evaluations per step. Unlike current baselines, EPIC yields sampling distributions that are empirically well-aligned with the entropy of the underlying data distribution. Across creative writing and summarization tasks, EPIC consistently improves LM-as-judge preference win-rates over widely used decoding strategies. These preference gains are complemented by automatic metrics, showing that EPIC produces more diverse generations and more faithful summaries. We also evaluate EPIC on mathematical reasoning, where it outperforms all baselines.

Problem

• language models
• decoding
• text generation
• entropy
• coherence

Innovation

• entropy-aligned decoding
• EPIC
• aleatoric uncertainty
• Lazy Gumbel-Max sampling
• hyperparameter-free decoding