Where Reasoning Matters: Rethinking Latent Reasoning in Semantic ID-based Generative Recommendation

2026-07-14

AI Summary

This work addresses a key challenge in semantic ID-based generative recommendation: how to efficiently allocate limited implicit inference computation to balance accuracy and efficiency. The authors propose an Information-gain-aware dynamic inference framework (IBA), which quantifies the contribution of each semantic ID position to reducing uncertainty about the target item through position-level information gain. They observe that earlier positions typically carry more informative signals and leverage this insight to dynamically allocate inference steps. IBA introduces, for the first time, a learnable computation budget allocation mechanism guided by information gain, overcoming the limitations of conventional fixed-step inference. Experiments on multiple public datasets demonstrate that IBA significantly outperforms strong baselines, achieving superior trade-offs between recommendation accuracy and computational cost while maintaining or even improving accuracy.

Abstract

Semantic ID-based generative recommendation predicts an item by generating a short sequence of semantic ID tokens, where each token is produced autoregressively. Latent reasoning has recently been introduced to improve this process through additional hidden-state computation before each token decision. This raises a practical question: when one item is represented by a sequence of semantic ID tokens, should each token receive the same fixed number of latent refinement steps, or should these steps be allocated more effectively across positions? We study this question through position-wise information-gain (IG), which measures how much each semantic ID position reduces the uncertainty of the target item. We observe that earlier semantic ID positions usually provide higher information-gain, while later positions contribute less additional information. We further analyze that applying more refinement to high-IG positions tends to bring larger expected benefits. Based on this observation, we propose IBA, an Information-Gain Budget Allocation framework for semantic ID-based generative recommendation. IBA treats latent refinement steps as a limited computational resource and learns how to allocate them across semantic ID positions, assigning more refinement to informative positions and less to positions with smaller contribution. Experiments on multiple public datasets show that IBA consistently improves strong generative recommendation baselines and achieves a better accuracy--computation trade-off than fixed or poorly matched step allocations.

Problem

• generative recommendation
• semantic ID
• latent reasoning
• information gain
• computation allocation

Innovation

• latent reasoning
• semantic ID
• information gain
• budget allocation
• generative recommendation