Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models

2026-08-14

AI Summary

This study addresses the challenge of aligning generated content with user intent in large language and vision-language models by presenting a systematic review of decoding methods during the inference phase. The research categorizes emerging approaches into three paradigms: token-level guidance, sequence-level generation, and parallel acceleration, while establishing a dedicated resource repository. As the first comprehensive survey of decoding strategy evolution, this work elucidates the critical role of these methods in enhancing both generation efficiency and alignment quality. Furthermore, it outlines future research directions, providing essential theoretical foundations and practical references for optimizing model inference performance.

Abstract

Large language models (LLMs) and large vision-language models (LVLMs) have demonstrated impressive generative capabilities, yet ensuring their outputs align with user intent is still challenging. While most existing approaches address this issue at the training stage, inference-time approaches like decoding methods offer a more efficient and scalable solution. Decoding methods control model generation by guiding token-level selection, performing sequence-level generation, or generating tokens in parallel to accelerate the process. In this survey, we identify three emerging paradigms from recent works on decoding methods for LLMs and LVLMs, provide a systematic review of these methods, highlight ongoing challenges, and discuss potential future research directions. Our goal is to underscore the efficiency and effectiveness of decoding methods and offer a practical view of their applications. Paper lists and more resources on decoding methods for LLMs and LVLMs can be found at https://github.com/wang2226/Awesome-LLM-Decoding.

Problem

• Decoding Methods
• Large Language Models
• Vision-Language Models
• User Intent Alignment
• Inference-time Control

Innovation

• Decoding Methods
• Inference-time Alignment
• Vision-Language Models
• Taxonomy