Legally-Informed Explainable AI

2025-04-14

AI Summary

In high-stakes domains (e.g., healthcare, education, finance), AI explanations must be both actionable and contestable; however, existing methods frequently overlook how legal context shapes user behavior and liability attribution. Method: This paper introduces the “legally informed explainable AI” paradigm, proposing the first systematic framework for *legally aware explainability*. It differentially models legal information needs and action pathways for three key stakeholders—clinicians, patients, and regulators—integrating legal knowledge graphs, context-aware explanation generation, stakeholder-specific demand modeling, and human-AI co-design. Contribution/Results: The work yields implementable design principles and practical guidelines that significantly enhance the usability, defensibility, and liability clarity of AI explanations in real-world legal settings, thereby advancing accountable, contestable, and regulation-compliant AI governance.

Technology Category

• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Algorithmic accountability and transparency on the web
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Explanations for artificial intelligence (AI) systems are intended to support the people who are impacted by AI systems in high-stakes decision-making environments, such as doctors, patients, teachers, students, housing applicants, and many others. To protect people and support the responsible development of AI, explanations need to be actionable--helping people take pragmatic action in response to an AI system--and contestable--enabling people to push back against an AI system and its determinations. For many high-stakes domains, such as healthcare, education, and finance, the sociotechnical environment includes significant legal implications that impact how people use AI explanations. For example, physicians who use AI decision support systems may need information on how accepting or rejecting an AI determination will protect them from lawsuits or help them advocate for their patients. In this paper, we make the case for Legally-Informed Explainable AI, responding to the need to integrate and design for legal considerations when creating AI explanations. We describe three stakeholder groups with different informational and actionability needs, and provide practical recommendations to tackle design challenges around the design of explainable AI systems that incorporate legal considerations.

Problem

• Develop actionable AI explanations for high-stakes decisions
• Ensure AI explanations are contestable to protect user rights
• Integrate legal considerations into explainable AI system design

Innovation

• Legally-Informed Explainable AI design
• Actionable and contestable AI explanations
• Legal considerations for stakeholder needs