ALPBench: A Benchmark for Attribution-level Long-term Personal Behavior Understanding

2026-02-03

AI Summary

Existing recommendation systems struggle to model users’ long-term, fine-grained preferences and lack the ability to understand new items at the attribute level. To address this limitation, this work proposes replacing traditional item-level recommendation with attribute-combination prediction as the evaluation objective and introduces ALPBench—the first benchmark specifically designed for attribute-level modeling of long-term user behavior. ALPBench reformulates historical user interactions into natural language sequences, leveraging the reasoning and generalization capabilities of large language models (LLMs) to enable interpretable and verifiable modeling of multi-attribute interactions and persistent interests. Experimental results reveal significant limitations of current LLMs in predicting complex attribute combinations, thereby establishing a novel evaluation paradigm for personalized recommendation systems.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent advances in large language models have highlighted their potential for personalized recommendation, where accurately capturing user preferences remains a key challenge. Leveraging their strong reasoning and generalization capabilities, LLMs offer new opportunities for modeling long-term user behavior. To systematically evaluate this, we introduce ALPBench, a Benchmark for Attribution-level Long-term Personal Behavior Understanding. Unlike item-focused benchmarks, ALPBench predicts user-interested attribute combinations, enabling ground-truth evaluation even for newly introduced items. It models preferences from long-term historical behaviors rather than users'explicitly expressed requests, better reflecting enduring interests. User histories are represented as natural language sequences, allowing interpretable, reasoning-based personalization. ALPBench enables fine-grained evaluation of personalization by focusing on the prediction of attribute combinations task that remains highly challenging for current LLMs due to the need to capture complex interactions among multiple attributes and reason over long-term user behavior sequences.

Problem

• personalized recommendation
• long-term behavior
• attribute-level preference
• user modeling
• LLM evaluation

Innovation

• Attribution-level Personalization
• Long-term Behavior Modeling
• Natural Language User History
• Attribute Combination Prediction
• LLM-based Recommendation