Why am I seeing this: Democratizing End User Auditing for Online Content Recommendations

2024-10-07

arXiv.org

AI Summary

Contemporary recommender systems personalize content using user attributes or inferred data but suffer from poor explainability and auditability, hindering users’ ability to make informed privacy decisions and undermining algorithmic accountability. This paper introduces the first lightweight, privacy-preserving, end-user–oriented algorithmic auditing paradigm: an interactive sandbox that enables users to actively formulate hypotheses—via synthetically generated user profiles and behavioral data—and observe system responses (e.g., ad delivery) in real time, transforming black-box attribution into a verifiable hypothesis-testing process. The approach integrates synthetic data modeling, a user-facing sandbox interface, and an A/B-style response observation framework. A user study demonstrates significant improvements in users’ comprehension of recommendation logic, attribution accuracy, and privacy decision-making capability; in advertising scenarios, hypothesis validation success reached 92%.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Privacy
• Humans and AI: Intelligent User Interfaces

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Algorithmic accountability and transparency on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Personalized recommendation systems tailor content based on user attributes, which are either provided or inferred from private data. Research suggests that users often hypothesize about reasons behind contents they encounter (e.g.,"I see this jewelry ad because I am a woman"), but they lack the means to confirm these hypotheses due to the opaqueness of these systems. This hinders informed decision-making about privacy and system use and contributes to the lack of algorithmic accountability. To address these challenges, we introduce a new interactive sandbox approach. This approach creates sets of synthetic user personas and corresponding personal data that embody realistic variations in personal attributes, allowing users to test their hypotheses by observing how a website's algorithms respond to these personas. We tested the sandbox in the context of targeted advertisement. Our user study demonstrates its usability, usefulness, and effectiveness in empowering end-user auditing in a case study of targeting ads.

Problem

• Users cannot verify why they see specific content recommendations
• Opaque systems hinder privacy and informed decision-making
• Lack of tools for end-user auditing of algorithmic behavior

Innovation

• Interactive sandbox for user hypothesis testing
• Synthetic user personas simulate real variations
• Enables auditing of recommendation algorithms