Representation Fidelity:Auditing Algorithmic Decisions About Humans Using Self-Descriptions

2026-03-05

AI Summary

This work addresses the critical yet underexplored question of whether algorithmic decision-making faithfully reflects individuals’ self-perceptions. To this end, we introduce the concept of “representational fidelity” and propose the first auditing framework tailored to credit-granting contexts, which quantifies the semantic distance between individuals’ natural language self-descriptions and the representations used as algorithmic inputs. Leveraging natural language generation, semantic alignment analysis, and expert annotations, we construct the Loan-Granting Self-Representations Corpus 2025, comprising 30,000 self-narratives. Furthermore, we develop a generalizable taxonomy of representational mismatches, offering a novel dimension for evaluating algorithmic fairness and interpretability.

Technology Category

• Knowledge Representation and Reasoning: Preferences
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems

Abstract

This paper introduces a new dimension for validating algorithmic decisions about humans by measuring the fidelity of their representations. Representation Fidelity measures if decisions about a person rest on reasonable grounds. We propose to operationalize this notion by measuring the distance between two representations of the same person: (1) an externally prescribed input representation on which the decision is based, and (2) a self-description provided by the human subject of the decision, used solely to validate the input representation. We examine the nature of discrepancies between these representations, how such discrepancies can be quantified, and derive a generic typology of representation mismatches that determine the degree of representation fidelity. We further present the first benchmark for evaluating representation fidelity based on a dataset of loan-granting decisions. Our Loan-Granting Self-Representations Corpus 2025 consists of a large corpus of 30 000 synthetic natural language self-descriptions derived from corresponding representations of applicants in the German Credit Dataset, along with expert annotations of representation mismatches between each pair of representations.

Problem

• Representation Fidelity
• Algorithmic Decisions
• Self-Descriptions
• Representation Mismatch
• Human-Centered AI

Innovation

• Representation Fidelity
• Algorithmic Decision-Making
• Self-Description
• Representation Mismatch
• Fairness Benchmark