🤖 AI Summary
This work proposes an explainability framework grounded in principles from social science to enhance the trustworthiness and transparency of rule-based agents in everyday applications. The framework innovatively integrates a policy-violation penalty mechanism with counterfactual reasoning to systematically generate contrastive natural language explanations—such as “Had this action not been taken, adverse event X would have occurred.” The core reasoning logic is implemented using Answer Set Programming, complemented by Python-based modules for information extraction and natural language generation. User studies demonstrate that the generated explanations significantly improve human understanding of agent decisions, thereby validating both the effectiveness and comprehensibility of the proposed approach.
📝 Abstract
In the field of Artificial Intelligence, an agent is a system which is able to autonomously make decisions in order to reach a desired goal. As these systems grow more prevalent in our day-to-day lives, there has been an increased need to add explainability features which can provide an account for an agent's behavior. We therefore propose a framework that outlines how to produce comprehensible explanations for policy-aware agents, or agents which have rule-enforcing policies incorporated in their decision-making framework. This framework is designed using insights from the social sciences on how to produce good explanations. It is implemented in the Answer Set Programming language while using Python to assist with information extraction and natural-language translation. Because these agents incur penalties when violating policies, we are able to leverage these penalties to detect undesirable events in scenarios that are counterfactual to the agents' original actions. This lends itself to creating contrastive explanations (e.g., "the agent performed this action because, had it not, undesirable event X would have occurred."), which formulate the core component for our explainability framework. The framework is evaluated using a survey wherein human participants provide feedback on our program-generated explanations.