PRISMe: A Novel LLM-Powered Tool for Interactive Privacy Policy Assessment

2025-01-27

AI Summary

Users struggle to comprehend lengthy, complex privacy policies, hindering informed privacy decision-making. This paper introduces PRISMe, an LLM-powered browser extension that enhances understanding of core privacy practices through real-time policy parsing, generation of visual risk dashboards, and a natural-language-enabled conversational interface. We conduct the first systematic, mixed-methods evaluation (N=22) of an LLM-driven privacy tool and propose a co-design framework balancing explainability, trustworthiness, and usability. Results demonstrate significant improvements in users’ privacy awareness and policy comprehension speed. However, two critical challenges emerge: inconsistent LLM output reliability and difficulties in establishing user trust. From these findings, we derive six empirically grounded, actionable design implications for developing trustworthy, interpretable, and usable LLM-based privacy assistance tools.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Philosophy and Ethics of AI: Privacy & Security

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Privacy-enhancing technologies

Abstract

Protecting online privacy requires users to engage with and comprehend website privacy policies, but many policies are difficult and tedious to read. We present PRISMe (Privacy Risk Information Scanner for Me), a novel Large Language Model (LLM)-driven privacy policy assessment tool, which helps users to understand the essence of a lengthy, complex privacy policy while browsing. The tool, a browser extension, integrates a dashboard and an LLM chat. One major contribution is the first rigorous evaluation of such a tool. In a mixed-methods user study (N=22), we evaluate PRISMe's efficiency, usability, understandability of the provided information, and impacts on awareness. While our tool improves privacy awareness by providing a comprehensible quick overview and a quality chat for in-depth discussion, users note issues with consistency and building trust in the tool. From our insights, we derive important design implications to guide future policy analysis tools.

Problem

• Privacy Policies
• User Comprehension
• Online Privacy Protection

Innovation

• PRISMe
• Visual Information Presentation
• Interactive Chat Functionality