OneLA: Scaling Linear-Attention Decoding to Large Beams in Generative Recommendation

2026-09-10

AI Summary

OneLA通过利用共享提示和短差异后缀，解决了生成式推荐中大束解码的线性注意力扩展问题，减少了内存使用和数据移动。

Abstract

Generative recommendation (GR) relies on large-beam decoding to generate hundreds of candidate items, creating a new scaling challenge for recurrent linear attention. Existing linear attention serving systems either materialize a full recurrent state for every beam or repeatedly replay shared history, incurring substantial memory and traffic overhead. To address this, we present OneLA, a linear-attention decoding framework that exploits the shared prompt and short divergent suffixes of GR workloads. Specifically, OneLA represents all beam states using a single shared prompt-derived state and compact, append-only records of their divergent transitions. Using this representation, OneLA computes only the state information required at each decoding step, without reconstructing a full recurrent state for every beam. Furthermore, OneLA uses a lightweight ancestry index to track the transition records that make up each beam's history, allowing beams to be updated without moving or copying existing records. A fused GPU kernel further reuses the shared state across beams. Our analysis shows that OneLA achieves 1.54-2.46x end-to-end decode speedups while substantially reducing recurrent-state memory use and data movement.

Problem

• generative recommendation
• large-beam decoding
• linear attention
• memory overhead
• data movement

Innovation

• linear-attention decoding
• shared prompt-derived state
• append-only records
• lightweight ancestry index
• fused GPU kernel