Comparative Personalization for Multi-document Summarization

2025-09-25

AI Summary

This work addresses two key challenges in multi-document summarization: (1) difficulty in modeling fine-grained inter-user preference differences, and (2) lack of standardized metrics for personalized evaluation. To this end, we propose ComPSum—a framework that learns structured user representations via inter-user preference contrastive learning, enabling dual personalization in writing style and content focus. We further introduce AuthorMap, a reference-free evaluation method that quantifies personalization quality through authorship attribution. Experiments on our newly constructed dataset, PerMSum, demonstrate that ComPSum significantly outperforms strong baselines in both ROUGE scores and human evaluations. AuthorMap achieves high correlation with human judgments (ρ > 0.82), confirming its validity and robustness as a reference-free personalized assessment metric. Our core contributions are twofold: (i) the first application of preference-aware contrastive learning to personalized summarization, and (ii) the establishment of a reliable, reference-free evaluation paradigm for personalized summarization.

Technology Category

• Natural Language Processing: Summarization
• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Data Visualization & Summarization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

Personalized multi-document summarization (MDS) is essential for meeting individual user preferences of writing style and content focus for summaries. In this paper, we propose that for effective personalization, it is important to identify fine-grained differences between users' preferences by comparing the given user's preferences with other users' preferences.Motivated by this, we propose ComPSum, a personalized MDS framework. It first generates a structured analysis of a user by comparing their preferences with other users' preferences. The generated structured analysis is then used to guide the generation of personalized summaries. To evaluate the performance of ComPSum, we propose AuthorMap, a fine-grained reference-free evaluation framework for personalized MDS. It evaluates the personalization of a system based on the authorship attribution between two personalized summaries generated for different users. For robust evaluation of personalized MDS, we construct PerMSum, a personalized MDS dataset in the review and news domain. We evaluate the performance of ComPSum on PerMSum using AuthorMap, showing that it outperforms strong baselines.

Problem

• Compares user preferences to identify fine-grained differences
• Generates structured user analysis to guide summary personalization
• Evaluates personalization via authorship attribution between user summaries

Innovation

• Compares user preferences for fine-grained personalization
• Generates structured user analysis to guide summarization
• Uses authorship attribution for reference-free evaluation