Exploration on Demand: From Algorithmic Control to User Empowerment

2025-07-29

AI Summary

Over-specialization in recommender systems fosters information silos, undermining content diversity and serendipity. To address this, we propose a user-controllable dynamic balancing framework. First, we design an adaptive clustering model that integrates Sentence-Transformer embeddings with online dynamic threshold clustering to enhance inter-cluster separability. Second, we introduce a user-driven exploration mechanism enabling real-time adjustment of diversity preferences. Third, we employ large language model (LLM)-powered synthetic user A/B testing to evaluate long-term preference evolution. Evaluated on MovieLens, our method reduces intra-list similarity from 0.34 to 0.26, increases serendipity to 0.73, and yields significantly higher preference for exploratory recommendations among 72.7% of synthetic users. Our core contribution lies in achieving interpretable, user-controllable co-optimization of personalization and diversity—bridging algorithmic transparency with actionable user agency.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Recommender Systems
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Recommender systems often struggle with over-specialization, which severely limits users' exposure to diverse content and creates filter bubbles that reduce serendipitous discovery. To address this fundamental limitation, this paper introduces an adaptive clustering framework with user-controlled exploration that effectively balances personalization and diversity in movie recommendations. Our approach leverages sentence-transformer embeddings to group items into semantically coherent clusters through an online algorithm with dynamic thresholding, thereby creating a structured representation of the content space. Building upon this clustering foundation, we propose a novel exploration mechanism that empowers users to control recommendation diversity by strategically sampling from less-engaged clusters, thus expanding their content horizons while preserving relevance. Experiments on the MovieLens dataset demonstrate the system's effectiveness, showing that exploration significantly reduces intra-list similarity from 0.34 to 0.26 while simultaneously increasing unexpectedness to 0.73. Furthermore, our Large Language Model-based A/B testing methodology, conducted with 300 simulated users, reveals that 72.7% of long-term users prefer exploratory recommendations over purely exploitative ones, providing strong evidence for the system's ability to promote meaningful content discovery without sacrificing user satisfaction.

Problem

• Over-specialization limits content diversity in recommender systems
• Balancing personalization and diversity in movie recommendations
• User-controlled exploration to reduce filter bubbles

Innovation

• Adaptive clustering framework balances personalization and diversity
• Sentence-transformer embeddings create semantically coherent clusters
• User-controlled exploration mechanism samples from less-engaged clusters