Comparative Analysis of Low-Rank Adaptation in Large Language Models versus Dense Embedding Regression for Headline Click-Through Rate Prediction

2026-08-12

AI Summary

This study addresses the problem of accurately selecting, from multiple candidate headlines, the one that achieves the highest click-through rate (CTR). The task is formulated as a winner-takes-all classification problem, and the work presents the first systematic comparison between parameter-efficiently fine-tuned large language models—specifically, a LoRA-finetuned causal language model (LOLAQwen-0.6B)—and lightweight dense embedding regression models. Evaluation is conducted on 3,263 real-world A/B test groups. Experimental results show that the embedding regression model attains a Top-1 accuracy of 42.79%, significantly outperforming LOLAQwen’s 35.70%. These findings demonstrate the effectiveness and advantages of discriminative lightweight models in high-throughput content ranking scenarios.

Abstract

Optimizing digital content headlines for click-through rate (CTR) is an important problem in online media and recommendation systems. While large language models (LLMs) have demonstrated strong generative capabilities, their effectiveness for discriminative ranking tasks, such as selecting the highest-performing headline from a set of candidates, remains less well understood. In this work, we compare a LoRA-fine-tuned causal language model, LOLAQwen (0.6B), with a dense embedding regression model for headline selection. We formulate headline selection as a winner-take-all classification problem and evaluate both approaches using a dataset of 3,263 A/B-tested headline groups. Performance is measured using Top-1 accuracy, defined as the proportion of groups for which the model correctly identifies the highest-performing headline. The embedding regression model achieves a Top-1 accuracy of 42.79%, compared with 35.70% for the LoRA-fine-tuned language model. These results indicate that, for this headline selection task, a lightweight discriminative approach can outperform a small generative language model fine-tuned using parameter-efficient adaptation. The findings highlight the potential of embedding-based regression models as efficient alternatives to generative models for high-throughput content ranking applications.

Problem

• headline selection
• click-through rate prediction
• CTR optimization
• content ranking
• discriminative ranking

Innovation

• Low-Rank Adaptation
• Embedding Regression
• Click-Through Rate Prediction
• Headline Selection
• Discriminative Ranking