Controlled Personalization in Legacy Media Online Services: A Case Study in News Recommendation

2025-10-10

AI Summary

Traditional news media face a tension between personalized recommendation and editorial values: how to enhance user engagement and content discovery efficiency without compromising journalistic integrity? This study proposes a “controlled personalization” framework that integrates human editorial judgment into algorithmic recommendation pipelines, enabling human–AI collaborative and controllable adaptation. Conducting an A/B test on a real-world news platform, we evaluate performance across multiple dimensions—click-through rate (CTR), navigation paths, and content diversity. Results show that moderate personalization significantly improves CTR (+18.3%), reduces browsing cost (22.7% fewer page transitions), and enhances content coverage and long-tail exposure, effectively mitigating popularity bias. Our core contribution is the first news recommendation paradigm that jointly ensures editorial autonomy, algorithmic interpretability, and value alignment.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Search and Optimization: Algorithm Configuration
• Machine Learning: Learning Preferences or Rankings

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Personalized news recommendations have become a standard feature of large news aggregation services, optimizing user engagement through automated content selection. In contrast, legacy news media often approach personalization cautiously, striving to balance technological innovation with core editorial values. As a result, online platforms of traditional news outlets typically combine editorially curated content with algorithmically selected articles - a strategy we term controlled personalization. In this industry paper, we evaluate the effectiveness of controlled personalization through an A/B test conducted on the website of a major Norwegian legacy news organization. Our findings indicate that even a modest level of personalization yields substantial benefits. Specifically, we observe that users exposed to personalized content demonstrate higher click-through rates and reduced navigation effort, suggesting improved discovery of relevant content. Moreover, our analysis reveals that controlled personalization contributes to greater content diversity and catalog coverage and in addition reduces popularity bias. Overall, our results suggest that controlled personalization can successfully align user needs with editorial goals, offering a viable path for legacy media to adopt personalization technologies while upholding journalistic values.

Problem

• Balancing editorial values with algorithmic personalization in news
• Evaluating controlled personalization effectiveness through A/B testing
• Reducing popularity bias while maintaining content diversity

Innovation

• Combines editorial curation with algorithmic selection
• Evaluates effectiveness through A/B testing methodology
• Balances user engagement with editorial values