ELEC: Efficient Large Language Model-Empowered Click-Through Rate Prediction

2025-09-09

AI Summary

CTR prediction in online advertising faces a fundamental trade-off between modeling collaborative signals and understanding textual semantics: conventional tabular models lack semantic awareness, while large language models (LLMs) struggle to efficiently capture user-item collaborative patterns and incur high inference latency. To address this, we propose a model-agnostic LLM-augmented framework based on a lightweight pseudo-siamese architecture: one stream processes structured features to model collaborative signals, while the other integrates LLM-derived text embeddings to enhance semantic understanding. We further introduce a joint knowledge distillation and collaborative filtering fusion strategy to drastically reduce LLM computational overhead. Evaluated on real-world industrial datasets, our method achieves significant AUC gains over state-of-the-art baselines while maintaining sub-millisecond inference latency—demonstrating superior effectiveness, efficiency, and deployment practicality.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Click-through rate (CTR) prediction plays an important role in online advertising systems. On the one hand, traditional CTR prediction models capture the collaborative signals in tabular data via feature interaction modeling, but they lose semantics in text. On the other hand, Large Language Models (LLMs) excel in understanding the context and meaning behind text, but they face challenges in capturing collaborative signals and they have long inference latency. In this paper, we aim to leverage the benefits of both types of models and pursue collaboration, semantics and efficiency. We present ELEC, which is an Efficient LLM-Empowered CTR prediction framework. We first adapt an LLM for the CTR prediction task. In order to leverage the ability of the LLM but simultaneously keep efficiency, we utilize the pseudo-siamese network which contains a gain network and a vanilla network. We inject the high-level representation vector generated by the LLM into a collaborative CTR model to form the gain network such that it can take advantage of both tabular modeling and textual modeling. However, its reliance on the LLM limits its efficiency. We then distill the knowledge from the gain network to the vanilla network on both the score level and the representation level, such that the vanilla network takes only tabular data as input, but can still generate comparable performance as the gain network. Our approach is model-agnostic. It allows for the integration with various existing LLMs and collaborative CTR models. Experiments on real-world datasets demonstrate the effectiveness and efficiency of ELEC for CTR prediction.

Problem

• Integrating LLMs with CTR models for better accuracy
• Reducing inference latency while maintaining semantic understanding
• Enabling efficient knowledge distillation between network architectures

Innovation

• LLM-adapted CTR prediction with pseudo-siamese network
• Knowledge distillation from gain to vanilla network
• Model-agnostic integration with LLMs and CTR models