🤖 AI Summary
To address memory inefficiency caused by fixed high-dimensional embeddings in recommender systems, this paper proposes a memory-constrained continuous embedding dimension optimization framework. Unlike conventional approaches that employ uniform high-dimensional embeddings or existing reinforcement learning (RL)-based methods limited to discrete dimension selection, our work introduces the first continuous-space embedding dimension search paradigm. We design a stochastic walk-driven exploration strategy to efficiently navigate the continuous dimension space, enabling joint optimization of recommendation accuracy and memory efficiency. The method is model-agnostic and plug-and-play. Extensive experiments on two real-world datasets and three state-of-the-art recommendation models demonstrate that our approach achieves superior performance across multiple memory budgets, consistently outperforming discrete-search baselines and establishing new state-of-the-art results.
📝 Abstract
Latent factor models are the most popular backbones for today's recommender systems owing to their prominent performance. Latent factor models represent users and items as real-valued embedding vectors for pairwise similarity computation, and all embeddings are traditionally restricted to a uniform size that is relatively large (e.g., 256-dimensional). With the exponentially expanding user base and item catalog in contemporary e commerce, this design is admittedly becoming memory-inefficient. To facilitate lightweight recommendation, reinforcement learning (RL) has recently opened up opportunities for identifying varying embedding sizes for different users/items. However, challenged by search efficiency and learning an optimal RL policy, existing RL-based methods are restricted to highly discrete, predefined embedding size choices. This leads to a largely overlooked potential of introducing finer granularity into embedding sizes to obtain better recommendation effectiveness under a given memory budget. In this paper, we propose continuous input embedding size search (CIESS), a novel RL-based method that operates on a continuous search space with arbitrary embedding sizes to choose from. In CIESS, we further present an innovative random walk-based exploration strategy to allow the RL policy to efficiently explore more candidate embedding sizes and converge to a better decision. CIESS is also model-agnostic and hence generalizable to a variety of latent factor RSs, whilst experiments on two real-world datasets have shown state-of-the-art performance of CIESS under different memory budgets when paired with three popular recommendation models.
