Repeated Padding for Sequential Recommendation

2024-03-11

ACM Conference on Recommender Systems

AI Summary

In conventional sequential recommendation, fixed-length padding with zero values leads to underutilized input space and information loss. To address this, we propose RepPad—a parameter-free, plug-and-play padding strategy that cyclically reuses the user’s historical interaction sequence for padding, thereby endowing padding positions with meaningful sequential information without introducing extra parameters or modifying model architecture. RepPad is fully compatible with mainstream sequential recommenders (e.g., GRU4Rec, SASRec) and integrates seamlessly into standard batch-wise training. Extensive experiments on five real-world benchmark datasets demonstrate consistent improvements: +60.3% average gain in HR@10 for GRU4Rec and +24.3% for SASRec, alongside accelerated convergence and enhanced generalization. To our knowledge, this is the first work to leverage padding positions as carriers of informative sequence content in sequential recommendation.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Sequential Decision Making
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Sequential recommendation aims to provide users with personalized suggestions based on their historical interactions. When training sequential models, padding is a widely adopted technique for two main reasons: 1) The vast majority of models can only handle fixed-length sequences; 2) Batch-based training needs to ensure that the sequences in each batch have the same length. The special value 0 is usually used as the padding content, which does not contain the actual information and is ignored in the model calculations. This common-sense padding strategy leads us to a problem that has never been explored in the recommendation field: Can we utilize this idle input space by padding other content to improve model performance and training efficiency further? In this paper, we propose a simple yet effective padding method called Repeated Padding (RepPad). Specifically, we use the original interaction sequences as the padding content and fill it to the padding positions during model training. This operation can be performed a finite number of times or repeated until the input sequences’ length reaches the maximum limit. Our RepPad can be considered as a sequence-level data augmentation strategy. Unlike most existing works, our method contains no trainable parameters or hyperparameters and is a plug-and-play data augmentation operation. Extensive experiments on various categories of sequential models and five real-world datasets demonstrate the effectiveness and efficiency of our approach. The average recommendation performance improvement is up to 60.3% on GRU4Rec and 24.3% on SASRec. We also provide in-depth analysis and explanation of what makes RepPad effective from multiple perspectives. Our datasets and codes are available at https://github.com/KingGugu/RepPad.

Problem

• Utilizing idle padding space to enhance sequential recommendation performance
• Proposing a padding-based data augmentation method without extra parameters
• Improving model efficiency and accuracy via repeated sequence padding

Innovation

• Uses original sequences as padding content
• Plug-and-play data augmentation operation
• Improves performance without trainable parameters