An Event is Worth One Token: Event Tokenization for Industrial-scale LLM Recommendation

2026-08-26

AI Summary

本文提出了一种事件为中心的方法，通过AMBER将每个交互的完整时间快照压缩为一个紧凑的事件令牌，以解决大规模语言模型推荐中信息丢失问题。

Abstract

LLM-based recommendation has scaled along model capacity and sequence length, yet each position encodes only text, semantic IDs, or a few categorical features, discarding rich user, item, context, and outcome signals available at each event. Under autoregressive modeling, this yields weak queries at each position and, since each position becomes context for the next, the degradation compounds across the sequence. We propose an event-centric paradigm that represents each interaction by its full temporal snapshot, and identify a new scaling dimension we term snapshot resolution: the amount of information encoded per event. To efficiently scale snapshot resolution, we introduce AMBER (Autoregressive Modeling via Bottlenecked Event Representation), which compresses each temporal snapshot into a compact Event Token, a new LLM input modality. The representation is learned end-to-end, while Event Tokens are pre-computed and cached for serving, decoupling snapshot resolution from real-time serving compute. On industrial-scale ranking and retrieval benchmarks, AMBER advances the compute-quality Pareto frontier relative to alternative recommendation paradigms. At sufficient capacity, a single unified tokenizer even outperforms dedicated per-entity tokenizers, demonstrating positive transfer across structurally different entity types. AMBER's Event Tokens also transfer across model architectures: when integrated into a heavily optimized non-LLM ranker as serving-time historical features, they yield statistically significant improvements. Further scaling Event Tokenizer capacity provides additional improvements.

Problem

• Event Tokenization
• LLM Recommendation
• Temporal Snapshot
• Autoregressive Modeling
• Snapshot Resolution

Innovation

• Event Tokenization
• Snapshot Resolution
• Autoregressive Modeling
• AMBER