Off-Policy Evaluation for Semantic ID Recommenders: Does the Model's Own Code Hierarchy Help?

📅 2026-08-28
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
研究解决了生成推荐系统中离线策略评估的问题,通过利用模型自身的语义ID树结构进行聚类来提高评估准确性。
📝 Abstract
Generative recommenders increasingly emit semantic IDs (SIDs): each item is a short sequence of hierarchical discrete codes from a residual quantizer, decoded autoregressively. Before spending scarce A/B-test, a team may decide offline which decoder or reranking variants are worth testing - a job for off-policy evaluation (OPE). We ask a simple question: can the model's own SID tree serve as the action abstraction for that OPE? Our answer has three parts. (i) Under the near-argmax logging real recommenders use, per-item OPE is hopeless - as item-level effective sample size is usually small on production logs - but marginalizing items to code-prefix clusters restores estimable support and cuts error. (ii) This gain is thanks to coarsening, not to the hierarchy specifically; but the SID tree is what makes coarsening feasible in a generative system - each cluster's mass is exactly and cheaply returned by the decoder, whereas flat clustering requires enumerating item/leaf masses that a code-only decoder does not directly expose. (iii) Resolution depth is the operative knob - coarser under scarce support - and a conditional bias bound links the coarsening bias to the quantizer's worst-case reconstruction residual and the target-logging divergence.
Problem

Research questions and friction points this paper is trying to address.

Off-Policy Evaluation
Semantic ID
Recommenders
Innovation

Methods, ideas, or system contributions that make the work stand out.

off-policy evaluation
semantic ID tree
code-prefix clustering
coarsening
resolution depth
🔎 Similar Papers
No similar papers found.
A
Artem Betlei
Criteo AI Lab, Grenoble, France