QueryFormer: Winning Solution for KDD Cup 2026 Tencent UniRec Challenge

2026-09-14

AI Summary

本文提出QueryFormer，通过交叉注意力生成查询并整合序列查询，有效解决了pCVR预测中特征交互与用户行为序列建模的问题。

Abstract

Post-click conversion rate (pCVR) prediction requires jointly modeling feature interactions and sequential user behaviors. The KDD Cup 2026 Tencent UniRec Challenge calls for a unified architecture addressing both. We observe that existing unified architectures often generate query tokens---the central information hub---with projection-based multi-layer perceptrons (MLPs), without explicit token-to-query attention for refining the query side. We propose QueryFormer, centered on a stackable unified field--sequence block that bridges non-sequential multi-field features and behavioral sequences, and provide a latency-aware scaling study over view width $H$, model width, depth, data, and compute. The block generates queries through cross-attention and packs sequence queries into shared-parameter attention. QueryFormer secured 1st place in the Industrial Track, achieving an official test area under the ROC curve (AUC) of 0.83254; a modest post-competition scale-up reached 0.832713. Within our grid, $H$-scaling improves validation AUC from 0.84540 to 0.84615 and beats HyFormer at comparable budgets. Ablation identifies query generation as the largest contributor. Packed shared-parameter cross-attention keeps H=8 inference latency to only 1.89x that of H=1, positioning the bridge as an efficient stackable unified block.

Problem

• pCVR
• feature interactions
• sequential user behaviors
• unified architecture
• query tokens

Innovation

• QueryFormer
• cross-attention
• unified field-sequence block
• packed shared-parameter attention
• latency-aware