Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

2026-08-23

AI Summary

本文提出了一种名为DRY的方法，在生成文本时通过调整logit来防止语言模型重复已出现的片段，从而减少循环并提高词汇多样性。

Abstract

Large Language Models generate text autoregressively, but open-ended generation is prone to verbatim looping, in which models repeat spans already present in context. Standard defenses such as repetition, presence, and frequency penalties and n-gram blocking act on token recurrence rather than the sequential structure of a loop, and often suppress looping only at strengths that also degrade formatting or fluency. We propose Don't Repeat Yourself (DRY), a sampling-time logit adjustment that penalizes a candidate token only when generating it would extend the current suffix into an exact continuation of a span seen earlier in the context. Sequence breakers protect chat templates and formatting tokens. Across models from 1.5B to 120B parameters, nine prompt families, and a 600-pair human study, DRY reduces suffix-extension rate by 47% while improving lexical diversity. An intervention-matched placebo produces no comparable reduction, identifying suffix matching as the operative mechanism. On AWQ-quantized 70B and 120B models, DRY reduces loop rate by roughly half while preserving MT-Bench, MMLU, and GSM8K performance, whereas standard alternatives lose measurable ground. DRY has been adopted by popular open-source LLM inference frameworks including llama.cpp, ExLlamaV2, and text-generation-webui, highlighting its practical impact on text generation.

Problem

• verbatim looping
• autoregressive generation
• token recurrence
• formatting degradation

Innovation

• Don't Repeat Yourself
• sampling-time logit adjustment
• verbatim looping prevention
• suffix-extension penalty
• lexical diversity improvement