Sustainability Evaluation Metrics for Recommender Systems

2025-07-30

AI Summary

Conventional recommendation system evaluation metrics—such as accuracy, recall, and user satisfaction—emphasize technical performance and short-term user experience, yet fail to capture long-term external impacts on environmental, social, and economic sustainability. Method: This paper pioneers the systematic integration of the United Nations Sustainable Development Goals (SDGs) into recommendation evaluation, proposing a tri-dimensional sustainability assessment framework grounded in environmental, social, and economic pillars. We design a scalable, operational, multi-level metric system aligned with SDGs and employ multi-dimensional modeling to quantify the externalities of recommendation behaviors. Contribution/Results: Through cross-scenario case studies, we empirically validate the framework’s applicability and effectiveness. Our work establishes both theoretical foundations and practical tools for transitioning recommendation systems from “accurate recommendation” to “responsible recommendation,” thereby advancing sustainable AI governance in real-world applications.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Evaluation and Analysis
• Multiagent Systems: Mechanism Design

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Sustainability-oriented evaluation metrics can help to assess the quality of recommender systems beyond wide-spread metrics such as accuracy, precision, recall, and satisfaction. Following the United Nations`s sustainable development goals (SDGs), such metrics can help to analyse the impact of recommender systems on environmental, social, and economic aspects. We discuss different basic sustainability evaluation metrics for recommender systems and analyze their applications.

Problem

• Assessing recommender systems beyond traditional accuracy metrics
• Evaluating impact on environmental, social, and economic sustainability
• Developing sustainability metrics aligned with UN SDGs

Innovation

• Sustainability-oriented evaluation metrics for recommenders
• Aligns with UN sustainable development goals (SDGs)
• Assesses environmental, social, economic impacts