HyMoERec: Hybrid Mixture-of-Experts for Sequential Recommendation

2025-11-09

AI Summary

Existing sequential recommendation models employ a uniform position-wise feed-forward network (FFN), overlooking the heterogeneity of user behavior patterns and the diversity of item complexity. To address this, we propose HyMoERec, the first sequential recommendation framework to incorporate the Mixture of Experts (MoE) mechanism. HyMoERec features a dual-branch architecture comprising shared experts and user- or item-specific experts, coupled with an adaptive fusion mechanism that enables differentiated modeling of heterogeneous behavioral dynamics and intricate item characteristics. This design enhances model expressiveness while preserving training stability. Extensive experiments on MovieLens-1M and Beauty demonstrate that HyMoERec consistently outperforms state-of-the-art baselines, achieving significant improvements in Recall@10 and NDCG@10. These results validate the effectiveness of explicitly modeling user and item heterogeneity through the MoE paradigm.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Data Mining & Knowledge Management: Recommender Systems
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

We propose HyMoERec, a novel sequential recommendation framework that addresses the limitations of uniform Position-wise Feed-Forward Networks in existing models. Current approaches treat all user interactions and items equally, overlooking the heterogeneity in user behavior patterns and diversity in item complexity. HyMoERec initially introduces a hybrid mixture-of-experts architecture that combines shared and specialized expert branches with an adaptive expert fusion mechanism for the sequential recommendation task. This design captures diverse reasoning for varied users and items while ensuring stable training. Experiments on MovieLens-1M and Beauty datasets demonstrate that HyMoERec consistently outperforms state-of-the-art baselines.

Problem

• Addresses uniform network limitations in sequential recommendation
• Captures heterogeneous user behavior and item diversity
• Improves recommendation accuracy through hybrid expert architecture

Innovation

• Hybrid mixture-of-experts architecture for sequential recommendation
• Combines shared and specialized expert branches
• Uses adaptive expert fusion mechanism for training