A Systematic Replicability and Comparative Study of BSARec and SASRec for Sequential Recommendation

2025-06-17

AI Summary

This study investigates the empirical effectiveness of frequency-aware enhancement mechanisms in sequential recommendation. Within the unified EasyRec framework, we rigorously reproduce and compare SASRec and BSARec under controlled experimental conditions—specifically standardizing bias term design, training configurations, and evaluation protocols to ensure fairness and reproducibility. Evaluation on benchmark datasets employs standard metrics: Recall@K and NDCG@K. Results confirm that BSARec’s frequency-aware bias term yields consistent performance gains; however, the observed improvements (+0.5–1.2%) are substantially smaller than those originally reported (+2.3–4.1%). This discrepancy underscores three key insights: (i) frequency-aware modeling provides measurable but diminishing returns; (ii) implementation details—particularly bias formulation and optimization settings—critically influence comparative outcomes; and (iii) methodological transparency and experimental reproducibility must be elevated as foundational norms in sequential recommendation research.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Recommender Systems
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

This study aims at comparing two sequential recommender systems: Self-Attention based Sequential Recommendation (SASRec), and Beyond Self-Attention based Sequential Recommendation (BSARec) in order to check the improvement frequency enhancement - the added element in BSARec - has on recommendations. The models in the study, have been re-implemented with a common base-structure from EasyRec, with the aim of obtaining a fair and reproducible comparison. The results obtained displayed how BSARec, by including bias terms for frequency enhancement, does indeed outperform SASRec, although the increases in performance obtained, are not as high as those presented by the authors. This work aims at offering an overview on existing methods, and most importantly at underlying the importance of implementation details for performance comparison.

Problem

• Compare BSARec and SASRec for sequential recommendation
• Evaluate frequency enhancement impact on recommendation performance
• Highlight implementation details' role in fair comparison

Innovation

• Comparing BSARec and SASRec for sequential recommendation
• Re-implementing models with EasyRec for fair comparison
• BSARec outperforms SASRec with frequency enhancement bias