HLLM-Creator: Hierarchical LLM-based Personalized Creative Generation

2025-08-25

AI Summary

Current AIGC systems struggle to fulfill users’ personalized creative needs—particularly in industrial advertising scenarios—where modeling fine-grained user interests under factual constraints, while balancing inference efficiency and data scarcity, remains challenging. Method: We propose a large-scale personalized ad headline generation framework featuring a hierarchical large language model (LLM) architecture. It integrates user clustering with advertisement-matching prediction-based pruning to accelerate inference, and introduces a novel Chain-of-Thought (CoT)-guided synthetic data pipeline to enhance personalized expression and factual consistency despite limited human annotations. Contribution/Results: Evaluated in online A/B tests on Douyin’s search advertising platform, our method achieves a statistically significant 0.476% lift in ad impression rate, demonstrating both technical efficacy and robust industrial deployability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

AI-generated content technologies are widely used in content creation. However, current AIGC systems rely heavily on creators' inspiration, rarely generating truly user-personalized content. In real-world applications such as online advertising, a single product may have multiple selling points, with different users focusing on different features. This underscores the significant value of personalized, user-centric creative generation. Effective personalized content generation faces two main challenges: (1) accurately modeling user interests and integrating them into the content generation process while adhering to factual constraints, and (2) ensuring high efficiency and scalability to handle the massive user base in industrial scenarios. Additionally, the scarcity of personalized creative data in practice complicates model training, making data construction another key hurdle. We propose HLLM-Creator, a hierarchical LLM framework for efficient user interest modeling and personalized content generation. During inference, a combination of user clustering and a user-ad-matching-prediction based pruning strategy is employed to significantly enhance generation efficiency and reduce computational overhead, making the approach suitable for large-scale deployment. Moreover, we design a data construction pipeline based on chain-of-thought reasoning, which generates high-quality, user-specific creative titles and ensures factual consistency despite limited personalized data. This pipeline serves as a critical foundation for the effectiveness of our model. Extensive experiments on personalized title generation for Douyin Search Ads show the effectiveness of HLLM-Creator. Online A/B test shows a 0.476% increase on Adss, paving the way for more effective and efficient personalized generation in industrial scenarios. Codes for academic dataset are available at https://github.com/bytedance/HLLM.

Problem

• Generating user-personalized content in AIGC systems
• Modeling user interests with factual constraints integration
• Ensuring efficiency and scalability for massive user bases

Innovation

• Hierarchical LLM framework for personalized content generation
• User clustering and matching-prediction pruning strategy
• Chain-of-thought reasoning data construction pipeline