🤖 AI Summary
This work addresses the limitations of traditional industrial recommendation systems, which rely on multi-stage cascaded architectures suffering from redundant feature processing, pipeline complexity, and excessively long service chains. The authors propose a unified generate-then-rank framework that encodes user history once and autoregressively generates semantic ID candidates, followed by item-level ranking based on shared encoder states. Innovatively, fine-grained preferences from a high-capacity teacher ranker are distilled into the single model via Rollout distillation, enhancing ranking quality without increasing online inference cost. Evaluated on Yandex Music at scale, the proposed model replaces a legacy cascade of over 15 components with comparable latency while significantly boosting active user count by 1.41%.
📝 Abstract
Industrial recommender systems are commonly deployed as multi-stage cascades with separate candidate generators, pre-rankers, and final rankers. Although effective, these cascades require repeated user-history processing, complex feature pipelines, and multiple serving stages. Semantic-ID-based generative retrieval offers a path toward simpler end-to-end systems, but next-item prediction alone does not capture the fine-grained preferences encoded by production ranking objectives. We present Gryphon-v2, a unified generate-and-rank architecture for end-to-end recommendation. The model encodes a user history once, generates Semantic-ID candidates with an autoregressive decoder, resolves them to catalogue items, and ranks them with an item-level Ranking Module that reuses the shared encoder states. To transfer fine-grained production ranking preferences without adding an expensive second model to the serving path, we distill a high-capacity, training-only Teacher Ranker into the Ranking Module. Gryphon-v2 is trained with Rollout Distillation: teacher scores are the only ranking supervision, and they are collected over two complementary candidate distributions. Rollouts from the current decoder expose the Ranking Module to candidates produced by the same generation mechanism used at serving time, while logged impressions cover items users were actually shown. In an online A/B experiment on a large-scale recommendation surface at Yandex Music, a single Gryphon-v2 model replaces a production cascade comprising more than 15 candidate generators, pre-ranking, and final ranking. The deployment increases the number of active users by 1.41% at serving latency comparable to the production cascade. These results support the practical viability of a generative retriever with a Ranking Module distilled from the Teacher Ranker as an end-to-end alternative to a production cascade.