Light-FMP: Lightweight Feature and Model Pruning for Enhanced Deep Recommender Systems

2026-05-07

AI Summary

Deep recommender systems often struggle to balance efficiency and accuracy under high-dimensional feature spaces. This work proposes Light-FMP, a novel framework that integrates, for the first time, a learnable masking mechanism based on the hard concrete distribution with a three-stage training strategy—pre-training, joint pruning, and continued fine-tuning. The approach first pre-trains the mask layer on a small data subset to identify critical features, then performs collaborative pruning of both model parameters and input features, and finally fine-tunes the pruned model on the full dataset. Light-FMP achieves joint lightweighting of features and model architecture, significantly outperforming existing methods across multiple real-world recommendation benchmarks. It substantially improves computational efficiency while maintaining or even enhancing recommendation accuracy, demonstrating strong scalability and robustness.

Abstract

Deep recommender systems (DRS) often face challenges in balancing computational efficiency and model accuracy, especially when handling high-dimensional input features. Existing methods either focus on improving accuracy while neglecting training efficiency or prioritize efficiency at the cost of suboptimal accuracy across tasks. We propose Light-FMP: Lightweight Feature and Model Pruning for Enhanced DRS, a lightweight framework that addresses the challenges through three key phases: \textit{pretraining}, \textit{pruning}, and \textit{continued training}. Using a hard concrete distribution, a masking layer is efficiently pretrained on a small data subset to identify important features. The model and features are then pruned, and training continues on the remaining dataset with domain-adapted parameters. Experiments on benchmark datasets from real-world recommender systems demonstrate that Light-FMP outperforms existing methods in both efficiency and accuracy while maintaining scalability and robustness.

Problem

• Deep Recommender Systems
• Computational Efficiency
• Model Accuracy
• High-dimensional Features

Innovation

• Lightweight pruning
• Feature and model co-pruning
• Hard concrete distribution
• Deep recommender systems
• Efficiency-accuracy trade-off