Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs

2026-08-23

AI Summary

本文针对dMLLMs生成长文本时出现的语义漂移和重复问题，提出了一种无需训练的解码方法CACD，通过结合软最大化置信度和邻近度来提高上下文相关性。

Abstract

Diffusion multimodal large language models (dMLLMs) frequently produce long-form outputs marred by semantic drift and repetition, with quality generally degrading as output length increases. We identify two structural deficiencies in existing decoding methods as primary drivers of these failures: confidence-based scoring ignores decoded-neighbor support, and block partitioning prevents access to high-readiness semantic anchors, together causing tokens to be committed before their local context is sufficiently established. We propose \ours{} (\textbf{C}ontext-\textbf{A}ware \textbf{C}luster \textbf{D}ecoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity, promoting contextually ready tokens above isolated candidates while suppressing low-confidence positional noise, operating block-free to keep high-readiness anchors globally accessible. \ours{} further applies architecture-aware calibration to handle confidence heterogeneity induced by diverse visual integration strategies. Experiments on three dMLLMs across four benchmarks demonstrate consistent quality gains and hallucination reduction over Original, with larger gains in several longer generation settings, highlighting the importance of neighbor support and visual integration strategy for future dMLLM decoding method design. Our code is openly available at https://github.com/zhaoyk-sysu/CACD-dMLLM.

Problem

• diffusion multimodal large language models
• semantic drift
• repetition
• output quality
• generation length

Innovation

• Context-Aware Cluster Decoding
• semantic drift
• neighbor proximity
• architecture-aware calibration
• visual integration strategy