MaRI: Accelerating Ranking Model Inference via Structural Re-parameterization in Large Scale Recommendation System

2026-02-26

AI Summary

This work addresses the inefficiency of inference in large-scale recommender systems, where redundant computations—particularly in feature fusion—often degrade performance without commensurate gains in accuracy. To tackle this challenge, the paper introduces structural reparameterization into ranking models for the first time, specifically optimizing matrix multiplication operations during feature fusion to eliminate user-side redundancy. The proposed approach achieves substantial acceleration of online inference while preserving model accuracy, thereby circumventing the typical trade-off between speed and performance. Furthermore, it complements existing efficiency-enhancing techniques such as model lightweighting and knowledge distillation, offering a practical and effective solution for improving serving efficiency in production environments.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Data Mining & Knowledge Management: Recommender Systems
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Ranking models, i.e., coarse-ranking and fine-ranking models, serve as core components in large-scale recommendation systems, responsible for scoring massive item candidates based on user preferences. To meet the stringent latency requirements of online serving, structural lightweighting or knowledge distillation techniques are commonly employed for ranking model acceleration. However, these approaches typically lead to a non-negligible drop in accuracy. Notably, the angle of lossless acceleration by optimizing feature fusion matrix multiplication, particularly through structural reparameterization, remains underexplored. In this paper, we propose MaRI, a novel Matrix Re-parameterized Inference framework, which serves as a complementary approach to existing techniques while accelerating ranking model inference without any accuracy loss. MaRI is motivated by the observation that user-side computation is redundant in feature fusion matrix multiplication, and we therefore adopt the philosophy of structural reparameterization to alleviate such redundancy.

Problem

• ranking model acceleration
• large-scale recommendation system
• structural re-parameterization
• accuracy loss
• feature fusion

Innovation

• structural re-parameterization
• lossless acceleration
• ranking model inference
• feature fusion
• recommendation system