Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy

2024-11-30

AI Summary

The relationship between model performance and data quality in sequential recommendation (SR) remains poorly understood, and conventional scaling laws fail to characterize SR systems accurately. Method: This paper establishes the first dedicated performance law for SR, introducing an approximate entropy (ApEn)-based metric to quantify data quality—moving beyond empirical reliance solely on dataset size. We systematically model the triadic relationship among performance, model scale, and data quality within Transformer-based architectures and perform regression analysis using HR and NDCG metrics to enable cross-dataset and cross-model-size performance prediction. Contribution/Results: Our law achieves prediction errors below 3.2% and attains a correlation coefficient of 0.91 between ApEn and empirical recommendation effectiveness. It supports optimal resource–performance trade-off decisions under arbitrary configuration settings, providing both theoretical foundations and practical tools for efficient deployment of SR systems in the large-model era.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Evaluation and Analysis
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Scaling Laws have emerged as a powerful framework for understanding how model performance evolves as they increase in size, providing valuable insights for optimizing computational resources. In the realm of Sequential Recommendation (SR), which is pivotal for predicting users' sequential preferences, these laws offer a lens through which to address the challenges posed by the scalability of SR models. However, the presence of structural and collaborative issues in recommender systems prevents the direct application of the Scaling Law (SL) in these systems. In response, we introduce the Performance Law for SR models, which aims to theoretically investigate and model the relationship between model performance and data quality. Specifically, we first fit the HR and NDCG metrics to transformer-based SR models. Subsequently, we propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics. Our method enables accurate predictions across various dataset scales and model sizes, demonstrating a strong correlation in large SR models and offering insights into achieving optimal performance for any given model configuration.

Problem

• Recommendation Systems
• Big Data
• Model Optimization

Innovation

• Sequence Recommendation Systems
• Approximate Entropy (ApEn)
• Performance Prediction