Distributed Download from an External Data Source in Asynchronous Faulty Settings

2025-09-03

AI Summary

This paper studies fault-tolerant downloading in asynchronous distributed data retrieval (DR): $k$ peer nodes, interconnected via an asynchronous P2P network, must each fully reconstruct a trusted external source $X$ of length $n gg k$, tolerating up to $eta k$ faulty nodes. It is the first work to extend DR to the asynchronous setting, providing novel protocols and tight bounds for both crash and Byzantine failures. For crash faults, we design the first deterministic optimal protocol tolerating any constant fraction $eta < 1$, achieving query complexity $Theta(n/k)$. For Byzantine faults, we prove that any randomized protocol requires $Omega(n)$ queries when $eta geq 1/2$, and construct a near-optimal randomized protocol for $eta < 1/2$. Our techniques integrate information diffusion, redundant querying, and lightweight consensus, jointly optimizing query efficiency and fault tolerance under asynchrony and strong adversarial conditions.

Technology Category

• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Large-scale security measurements

Abstract

The distributedData Retrieval (DR) model consists of $k$ peers connected by a complete peer-to-peer communication network, and a trusted external data source that stores an array $ extbf{X}$ of $n$ bits ($n gg k$). Up to $βk$ of the peers might fail in any execution (for $βin [0, 1)$). Peers can obtain the information either by inexpensive messages passed among themselves or through expensive queries to the source array $ extbf{X}$. In the DR model, we focus on designing protocols that minimize the number of queries performed by any nonfaulty peer (a measure referred to as query complexity) while maximizing the resilience parameter $β$. The Download problem requires each nonfaulty peer to correctly learn the entire array $ extbf{X}$. Earlier work on this problem focused on synchronous communication networks and established several deterministic and randomized upper and lower bounds. Our work is the first to extend the study of distributed data retrieval to asynchronous communication networks. We address the Download problem under both the Byzantine and crash failure models. We present query-optimal deterministic solutions in an asynchronous model that can tolerate any fixed fraction $β<1$ of crash faults. In the Byzantine failure model, it is known that deterministic protocols incur a query complexity of $Ω(n)$ per peer, even under synchrony. We extend this lower bound to randomized protocols in the asynchronous model for $βgeq 1/2$, and further show that for $β< 1/2$, a randomized protocol exists with near-optimal query complexity. To the best of our knowledge, this is the first work to address the Download problem in asynchronous communication networks.

Problem

• Distributed data retrieval in asynchronous faulty networks
• Minimizing query complexity while maximizing fault tolerance
• Handling both Byzantine and crash failure models

Innovation

• Asynchronous network distributed data retrieval
• Deterministic crash fault tolerant solutions
• Randomized Byzantine fault protocol optimization