Ad Headline Generation using Self-Critical Masked Language Model

2026-07-07

AI Summary

This work addresses the challenge of generating high-quality, engaging product titles at scale for e-commerce platforms by proposing a self-critical reinforcement learning approach based on Transformer-based masked language models. The method introduces self-critical policy gradient training into masked language modeling for the first time and enables joint conditional generation conditioned on multiple product attributes, effectively balancing grammatical correctness with creative diversity. Experimental results demonstrate that the proposed approach outperforms existing Transformer- and LSTM+RL-based baselines on both automatic overlap metrics and human evaluations, with generated titles even surpassing those crafted by human copywriters in terms of overall quality.

Abstract

For any E-commerce website it is a nontrivial problem to build enduring advertisements that attract shoppers. It is hard to pass the creative quality bar of the website, especially at a large scale. We thus propose a programmatic solution to generate product advertising headlines using retail content. We propose a state of the art application of Reinforcement Learning (RL) Policy gradient methods on Transformer based Masked Language Models. Our method creates the advertising headline by jointly conditioning on multiple products that a seller wishes to advertise. We demonstrate that our method outperforms existing Transformer and LSTM + RL methods in overlap metrics and quality audits. We also show that our model-generated headlines outperform human submitted headlines in terms of both grammar and creative quality as determined by audits.

Problem

• Ad Headline Generation
• E-commerce
• Creative Quality
• Large-scale Advertising
• Retail Content

Innovation

• Self-Critical Learning
• Masked Language Model
• Reinforcement Learning
• Transformer
• Ad Headline Generation