Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations

2026-03-01

AI Summary

Existing generative recommendation models commonly adopt a “flat sequence” assumption, neglecting the session-level hierarchical structure of user behavior, which limits representational capacity, reduces computational efficiency, and increases susceptibility to noise. To address this, this work proposes the Hierarchical Preference-Guided Generative Recommendation framework (HPGR), which uniquely integrates behavioral hierarchy and preference-aware mechanisms into generative recommendation. HPGR first performs structure-aware pre-training via session-aware masked item modeling to learn hierarchical semantic representations, then employs a preference-guided sparse attention mechanism for efficient fine-tuning. Evaluated on the large-scale industrial APPGallery dataset and online A/B tests, HPGR significantly outperforms strong baselines such as HSTU and MTGR, achieving state-of-the-art performance.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Natural Language Processing: Generation
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Generative Recommenders (GRs), exemplified by the Hierarchical Sequential Transduction Unit (HSTU), have emerged as a powerful paradigm for modeling long user interaction sequences. However, we observe that their "flat-sequence" assumption overlooks the rich, intrinsic structure of user behavior. This leads to two key limitations: a failure to capture the temporal hierarchy of session-based engagement, and computational inefficiency, as dense attention introduces significant noise that obscures true preference signals within semantically sparse histories, which deteriorates the quality of the learned representations. To this end, we propose a novel framework named HPGR (Hierarchical and Preference-aware Generative Recommender), built upon a two-stage paradigm that injects these crucial structural priors into the model to handle the drawback. Specifically, HPGR comprises two synergistic stages. First, a structure-aware pre-training stage employs a session-based Masked Item Modeling (MIM) objective to learn a hierarchically-informed and semantically rich item representation space. Second, a preference-aware fine-tuning stage leverages these powerful representations to implement a Preference-Guided Sparse Attention mechanism, which dynamically constrains computation to only the most relevant historical items, enhancing both efficiency and signal-to-noise ratio. Empirical experiments on a large-scale proprietary industrial dataset from APPGallery and an online A/B test verify that HPGR achieves state-of-the-art performance over multiple strong baselines, including HSTU and MTGR.

Problem

• Generative Recommenders
• flat-sequence assumption
• temporal hierarchy
• computational inefficiency
• preference signals

Innovation

• Hierarchical Recommendation
• Preference-Aware Attention
• Sparse Attention
• Generative Recommender
• Session-based Modeling