Efficient Decentralized Multi-task Dataset Valuation via Model Merging

2026-07-03

AI Summary

This work addresses the limitations of existing data valuation methods, which are largely confined to single-task settings and struggle to efficiently assess data contributions in decentralized, privacy-sensitive multi-task learning scenarios. The authors propose DMVM, a novel framework that, for the first time, integrates model fusion and task arithmetic into multi-task data valuation. DMVM enables inference of each dataset’s marginal contribution to multi-task performance through parameter-space operations—without requiring model retraining or sharing raw data—and supports decentralized collaboration via secure aggregation protocols. Theoretical analysis provides bounds on approximation error, and experiments on computer vision and natural language processing benchmarks demonstrate that DMVM significantly outperforms baseline approaches such as Shapley value estimation in terms of efficiency, scalability, and valuation accuracy.

Abstract

Accurate and efficient dataset valuation is essential for enabling fair and transparent data marketplaces, especially when multiple contributors provide data for training multi-task models. Most existing valuation methods, however, are limited to single-task settings, overlooking scenarios where a buyer aims to optimize performance across multiple downstream tasks. Moreover, traditional valuation approaches, such as Shapley-based or retraining-based methods, are computationally expensive and poorly suited for decentralized environments without a trusted central coordinator and with strict privacy constraints. We propose DMVM (Decentralized Multi-task Valuation via Model Merging), a novel framework that bypasses retraining and data sharing by leveraging task arithmetic to infer dataset contributions directly from model combinations. Instead of retraining or sharing raw data, DMVM quantifies how models trained on different datasets combine in parameter space to infer each dataset's marginal utility across multiple tasks. This formulation yields a valuation process that is scalable, computationally efficient, and explicitly aligned with multi-task generalization behavior. To support decentralized deployment, we introduce a secure aggregation protocol that enables collaborative valuation without revealing individual model parameters or private data. We also provide theoretical error bounds characterizing the approximation quality of DMVM and validate our framework through comprehensive experiments on computer vision and natural language processing tasks.

Problem

• dataset valuation
• multi-task learning
• decentralized
• privacy
• data marketplace

Innovation

• multi-task dataset valuation
• model merging
• decentralized learning
• task arithmetic
• privacy-preserving aggregation