Beyond Relevance: An Adaptive Exploration-Based Framework for Personalized Recommendations

2025-03-25

AI Summary

Recommender systems face a fundamental trade-off among personalization, diversity, and cold-start robustness in dynamic content environments. To address this, we propose an adaptive exploration framework featuring two key innovations: (1) online semantic clustering via Sentence-Transformer embeddings, enabling incremental cluster formation with an adaptive threshold that dynamically responds to evolving user preferences and content distributions; and (2) a user-controllable exploration mechanism grounded in empirical analysis linking interaction history to explicit diversity preferences. The framework ensures linearly scalable recommendation generation. Evaluated on MovieLens, it achieves intra-list similarity of 0.26, serendipity of 0.73, and a 72.7% long-term user acceptance rate for exploratory recommendations. Computational complexity grows linearly with the number of clusters, markedly enhancing cold-start robustness and sustained user engagement.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Applications of semantic technologies for improving search, browsing, recommendation, personalization

Abstract

Recommender systems must balance personalization, diversity, and robustness to cold-start scenarios to remain effective in dynamic content environments. This paper introduces an adaptive, exploration-based recommendation framework that adjusts to evolving user preferences and content distributions to promote diversity and novelty without compromising relevance. The system represents items using sentence-transformer embeddings and organizes them into semantically coherent clusters through an online algorithm with adaptive thresholding. A user-controlled exploration mechanism enhances diversity by selectively sampling from under-explored clusters. Experiments on the MovieLens dataset show that enabling exploration reduces intra-list similarity from 0.34 to 0.26 and increases unexpectedness to 0.73, outperforming collaborative filtering and popularity-based baselines. A/B testing with 300 simulated users reveals a strong link between interaction history and preference for diversity, with 72.7% of long-term users favoring exploratory recommendations. Computational analysis confirms that clustering and recommendation processes scale linearly with the number of clusters. These results demonstrate that adaptive exploration effectively mitigates over-specialization while preserving personalization and efficiency.

Problem

• Balancing personalization, diversity, and robustness in recommender systems
• Adapting to evolving user preferences and content distributions
• Mitigating over-specialization while preserving relevance and efficiency

Innovation

• Adaptive exploration-based recommendation framework
• Sentence-transformer embeddings for item representation
• Online clustering with adaptive thresholding