Beyond Models! Explainable Data Valuation and Metric Adaption for Recommendation

2025-02-12

AI Summary

To address the challenges of heterogeneous user behavioral data quality and the lack of interpretability and cross-metric generalizability in existing recommendation evaluation methods, this paper proposes DVR—a novel, interpretable, and general framework for data value assessment and metric-adaptive optimization. Its core contributions are: (1) the first Shapley-value-based data valuation mechanism, enabling model-agnostic and interpretable quantification of data quality; and (2) a reinforcement learning–driven metric adapter that unifies handling of both differentiable and non-differentiable evaluation objectives, effectively decoupling data valuation from metric-specific dependencies. Extensive experiments on multiple benchmark datasets demonstrate that DVR significantly improves recommendation performance—achieving up to a 34.7% gain in NDCG—while simultaneously enhancing ranking accuracy, diversity, and fairness.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Learning Human Values and Preferences
• Search and Optimization: Evaluation and Analysis

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

User behavior records serve as the foundation for recommender systems. While the behavior data exhibits ease of acquisition, it often suffers from varying quality. Current methods employ data valuation to discern high-quality data from low-quality data. However, they tend to employ black-box design, lacking transparency and interpretability. Besides, they are typically tailored to specific evaluation metrics, leading to limited generality across various tasks. To overcome these issues, we propose an explainable and versatile framework DVR which can enhance the efficiency of data utilization tailored to any requirements of the model architectures and evaluation metrics. For explainable data valuation, a data valuator is presented to evaluate the data quality via calculating its Shapley value from the game-theoretic perspective, ensuring robust mathematical properties and reliability. In order to accommodate various evaluation metrics, including differentiable and non-differentiable ones, a metric adapter is devised based on reinforcement learning, where a metric is treated as the reinforcement reward that guides model optimization. Extensive experiments conducted on various benchmarks verify that our framework can improve the performance of current recommendation algorithms on various metrics including ranking accuracy, diversity, and fairness. Specifically, our framework achieves up to 34.7% improvements over existing methods in terms of representative NDCG metric. The code is available at https://github.com/renqii/DVR.

Problem

• Enhances data valuation transparency in recommender systems.
• Adapts to diverse evaluation metrics using reinforcement learning.
• Improves recommendation algorithms' performance across multiple metrics.

Innovation

• Explainable data valuation via Shapley value
• Metric adapter using reinforcement learning
• Versatile framework for any model architectures