TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding

2026-07-13

AI Summary

This work addresses the limited acceleration efficacy of existing speculative decoding methods in multimodal large language models, which often neglect visual content relevance or fail to optimize the verifier’s accepted prefix length. To overcome these limitations, the authors propose TIGER, a novel framework that introduces a text-conditioned visual gating routing mechanism to dynamically select vision tokens relevant to the current generation state. TIGER further incorporates an alignment strategy trained specifically to maximize accepted prefix length. Through sparse selection of visual tokens, KL-anchored distillation warm-up, and group-based policy optimization, TIGER significantly enhances the synergy between the drafter and verifier. Experimental results demonstrate that TIGER achieves substantially longer accepted prefixes and higher speedup ratios while preserving downstream task accuracy, thereby offering a superior trade-off between generation quality and latency.

Abstract

Speculative decoding accelerates autoregressive generation by letting a lightweight drafter propose multiple tokens that are verified by a larger target model. Although effective for text-only LLMs, speculative decoding yields limited gains in VLMs because drafters often diverge on vision-critical content, while existing multimodal acceleration methods do not directly address irrelevant visual evidence or optimize the verifier-accepted prefix length that governs speedup. We propose TIGER, a Text-conditioned vIsual GatEd Routing framework for multimodal speculative decoding. TIGER dynamically selects a sparse set of context-relevant visual tokens based on the drafter's current textual state, rather than expose the full visual token set or a fixed compressed interface. To better align training with inference-time efficiency, we optimize the drafter with acceptance-aligned group-based policy training using verifier-derived rewards based on accepted prefix length, built on top of distillation warm start with KL anchoring. This encourages the drafter not only to imitate the target model, but also to produce speculative continuations that survive verification for longer prefixes. Experiments show that TIGER yields consistent gains in accepted prefix length and speculative speedup under exact verifier-side speculative decoding, while achieving favorable quality-latency trade-offs with comparable downstream accuracy in visual-routing analyses.

Problem

• speculative decoding
• vision-language models
• visual tokens
• prefix acceptance
• multimodal acceleration

Innovation

• speculative decoding
• multimodal language models
• visual token routing
• acceptance-aligned training
• gated routing