Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding

2026-06-21

AI Summary

This work addresses the tendency of large language models to produce degenerate, repetitive, and lexically monotonous outputs in open-ended generation—a phenomenon often attributed to the “likelihood trap” that diverges from human-like text patterns. The authors propose a training-free, pre-decoding intervention that combines a context spotlight mechanism to suppress global stop words while amplifying contextually salient tokens, with an adaptive debiasing strategy based on real-time logit standard deviation to enable scale-invariant repetition penalties. Additionally, pointwise mutual information (PMI)-guided contextual modulation is introduced, seamlessly integrating with mainstream decoding strategies such as Top-p and Min-p. Experimental results demonstrate consistent improvements across open-ended generation, factual question answering, and mathematical reasoning tasks, yielding substantial gains in textual diversity, coherence, and reasoning accuracy under high-temperature decoding, all with negligible computational overhead.

Abstract

In open-ended generation, LLMs frequently fall into the "likelihood trap", marked by repetitive degeneration and vocabulary dullness, creating a discrepancy between machine-generated and human-written text. While post-hoc tail truncation (e.g., Top-$p$, Min-$p$) avoids sampling from the unreliable tail, it can over-sample from the uncalibrated head and misalign generation with human lexical preferences; fixed scalar repetition penalties likewise ignore variation in logit scale across inference steps, potentially disrupting semantic coherence. To address both limitations, we propose Variance-Calibrated Modulation (VCM), a training-free pre-decoding intervention that reshapes the probability distribution before truncation through two dynamic mechanisms: (1) Contextual Searchlight via PMI, which suppresses global stopwords while elevating context-evoked tokens, and (2) Adaptive Self-Debiasing, which uses real-time logit standard deviation for scale-invariant penalization. Across open-ended generation, factual QA, and mathematical reasoning, VCM consistently mitigates the likelihood trap. With negligible computational overhead, VCM integrates with existing decoding strategies, improving diversity, coherence, and, particularly at higher decoding temperatures, reasoning accuracy.

Problem

• likelihood trap
• repetitive degeneration
• vocabulary dullness
• decoding bias
• logit scale variation

Innovation

• Variance-Calibrated Modulation
• likelihood trap
• contextual searchlight
• adaptive self-debiasing
• logit calibration