Can I Trust This Chatbot? Assessing User Privacy in AI-Healthcare Chatbot Applications

2025-09-17

AI Summary

This study empirically assesses privacy risks faced by users of AI-powered healthcare chatbots during the collection and processing of sensitive health data. We conducted a three-stage qualitative audit of 12 widely used applications from the U.S. App Store and Google Play, systematically examining (1) privacy settings during registration, (2) in-app data control mechanisms, and (3) textual analysis of privacy policies—thereby establishing the first multidimensional privacy evaluation framework integrating interface design, user agency, and legal documentation. Key findings include: 50% of apps fail to present privacy policies at registration; only two support opt-out of third-party data sharing; and most policies omit concrete technical or organizational data protection measures, severely constraining user data autonomy. This work fills a critical gap in empirical research on privacy practices in AI-driven healthcare dialogue systems and provides an actionable, evidence-based assessment benchmark for regulatory refinement and responsible product design.

Technology Category

• Philosophy and Ethics of AI: Privacy & Security
• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

As Conversational Artificial Intelligence (AI) becomes more integrated into everyday life, AI-powered chatbot mobile applications are increasingly adopted across industries, particularly in the healthcare domain. These chatbots offer accessible and 24/7 support, yet their collection and processing of sensitive health data present critical privacy concerns. While prior research has examined chatbot security, privacy issues specific to AI healthcare chatbots have received limited attention. Our study evaluates the privacy practices of 12 widely downloaded AI healthcare chatbot apps available on the App Store and Google Play in the United States. We conducted a three-step assessment analyzing: (1) privacy settings during sign-up, (2) in-app privacy controls, and (3) the content of privacy policies. The analysis identified significant gaps in user data protection. Our findings reveal that half of the examined apps did not present a privacy policy during sign up, and only two provided an option to disable data sharing at that stage. The majority of apps' privacy policies failed to address data protection measures. Moreover, users had minimal control over their personal data. The study provides key insights for information science researchers, developers, and policymakers to improve privacy protections in AI healthcare chatbot apps.

Problem

• Evaluating privacy practices in AI healthcare chatbot apps
• Assessing data protection gaps in user privacy controls
• Analyzing privacy policy content and data sharing options

Innovation

• Analyzed privacy settings during sign-up
• Evaluated in-app privacy controls functionality
• Assessed privacy policies content comprehensively