TransRetrieval: Scaling Up Transformer-Based Retrieval for Industrial Recommendation

2026-08-26

AI Summary

为解决特征异质性导致的Transformer层堆叠效率低下问题，提出TransRetrieval框架，通过加权平均聚合、目标令牌压缩及位置式域嵌入方法提高推荐检索性能。

Abstract

Applying scaling laws to recommendation retrieval is hindered by feature heterogeneity: naively stacking Transformer layers yields diminishing returns because heterogeneous fields produce severe token-norm divergence. We present TransRetrieval, a Transformer-based retrieval framework that scales with both computational budget and cross-domain data. The key enabler is (1) weighted average aggregation, which restores the homogeneous-token assumption Transformers rely on. Building on this, we introduce (2) target token compression that cuts per-candidate FLOPs by 85% while preserving cross-attention expressiveness, and (3) position-style domain embeddings that unify multiple domains at negligible additional cost, turning cross-domain data into a scaling asset. On a 40-billion-interaction industrial dataset and the public KuaiRand benchmark, scaling compute from 0.1 to 2 MFLOPs per target yields +19.3/+22.2 pt Recall@2000, confirming robust log-linear scaling. In online A/B tests, TransRetrieval lifts platform revenue by 2.53% under the same end-to-end latency constraint as the production baseline.

Problem

• feature heterogeneity
• Transformer-based retrieval
• recommendation system

Innovation

• weighted average aggregation
• target token compression
• position-style domain embeddings