From Fairness to Truthfulness: Rethinking Data Valuation Design

2025-04-07

AI Summary

This paper addresses incentive incompatibility in data markets arising from data owners’ private and heterogeneous costs, showing that mainstream valuation methods—such as Leave-One-Out and Data Shapley—fail to incentivize truthful cost reporting, thereby undermining market efficiency. To resolve this, we formally establish incentive compatibility as a fundamental design principle for data valuation and, grounded in mechanism design theory, propose Myerson-optimal and VCG payment rules tailored to data markets. We rigorously prove that the Myerson payment rule constitutes the minimal truthful mechanism and is buyer-optimal, and further demonstrate its equivalence to the VCG rule under unconstrained allocation. This work introduces the first data valuation framework with provable incentive compatibility, significantly improving transaction efficiency and fairness. It provides both a theoretically sound foundation and a practically deployable solution for trustworthy data trading.

Technology Category

• Game Theory and Economic Paradigms: Mechanism Design
• Multiagent Systems: Mechanism Design
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

As large language models increasingly rely on external data sources, fairly compensating data contributors has become a central concern. In this paper, we revisit the design of data markets through a game-theoretic lens, where data owners face private, heterogeneous costs for data sharing. We show that commonly used valuation methods--such as Leave-One-Out and Data Shapley--fail to ensure truthful reporting of these costs, leading to inefficient market outcomes. To address this, we adapt well-established payment rules from mechanism design, namely Myerson and Vickrey-Clarke-Groves (VCG), to the data market setting. We demonstrate that the Myerson payment is the minimal truthful payment mechanism, optimal from the buyer's perspective, and that VCG and Myerson payments coincide in unconstrained allocation settings. Our findings highlight the importance of incorporating incentive compatibility into data valuation, paving the way for more robust and efficient data markets.

Problem

• Ensuring truthful cost reporting in data markets
• Addressing inefficiency in current data valuation methods
• Designing incentive-compatible payment mechanisms for data sharing

Innovation

• Game-theoretic approach for data markets
• Myerson payment ensures truthful cost reporting
• VCG and Myerson payments coincide unconstrained settings