TailorMind: Towards Preference-Aligned Multimodal Content Generation

2026-06-22

AI Summary

This work addresses the challenge of generating high-quality, preference-aligned multimodal content in personalized systems when user-generated content (UGC) is scarce. To this end, we propose a novel paradigm that integrates collaborative preference modeling with controllable multimodal generation. Our approach leverages hypergraph collaborative filtering to enhance user representations under sparse interactions and refines user profiles via text gradient descent. High-fidelity generation is achieved through retrieval-augmented style control and a cross-modal consistency reflection mechanism. Evaluated on our newly curated benchmark, TailorBench, the proposed method substantially outperforms state-of-the-art generative models and real UGC across coherence, novelty, and aesthetic quality, while achieving up to a 29% relative improvement in recall on reranking tasks.

Abstract

Personalized content systems depend on available UGC and struggle when suitable content is absent, delayed, or costly to create. Although multimodal generators can synthesize content on demand, how to translate behavioral traces into generation-ready preferences remains underexplored. We study personalized multimodal content generation: creating user-tailored multimodal content without existing item pools or waiting for matching UGC. We propose TailorMind, linking collaborative preference modeling with controllable multimodal generation. TailorMind enriches sparse user histories via hypergraph collaborative filtering and optimizes textual profiles with ranking-error feedback and textual gradient descent. Retrieval-augmented style control grounds outputs in authentic UGC patterns, while cross-modal cohesion reflection reduces semantic drift. We construct TailorBench, a benchmark from three mainstream platforms evaluated along five dimensions: coherence, novelty, aesthetic, hallucination, profiling. Experiments show that TailorMind achieves competitive or stronger coherence, improves novelty and aesthetic quality over representative generation baselines and ground-truth UGC, demonstrating advantages over retrieving available content or comparable UGC, while achieving up to 29% Recall gains in reranking. Our code is released at: https://github.com/iLearn-Lab/TailorMind.

Problem

• personalized content generation
• multimodal generation
• preference alignment
• user behavior
• content synthesis

Innovation

• preference-aligned generation
• hypergraph collaborative filtering
• retrieval-augmented style control
• cross-modal cohesion
• personalized multimodal generation