DynoStore: A wide-area distribution system for the management of data over heterogeneous storage
🤖 AI Summary
To address inefficient data management in heterogeneous storage systems—caused by protocol diversity, fragmented authentication mechanisms, and the absence of a unified coordination framework—this paper proposes the Wide-area Data Distribution System (WDDS). WDDS introduces three key innovations: (1) a “data container” abstraction that unifies interfaces and access semantics across heterogeneous storage sources; (2) an elastic, scalable wide-area storage network integrating erasure coding with dynamic load balancing; and (3) a lightweight distributed authentication model natively supporting S3, POSIX, and WebDAV protocols. Evaluated on medical imaging and satellite remote sensing workloads, WDDS achieves a 10% higher throughput than centralized cloud solutions, faster failure recovery than Redis and IPFS, and sustained service delivery under >10,000 concurrent clients and large-scale node failures.
Application Category
📝 Abstract
Data distribution across different facilities offers benefits such as enhanced resource utilization, increased resilience through replication, and improved performance by processing data near its source. However, managing such data is challenging due to heterogeneous access protocols, disparate authentication models, and the lack of a unified coordination framework. This paper presents DynoStore, a system that manages data across heterogeneous storage systems. At the core of DynoStore are data containers, an abstraction that provides standardized interfaces for seamless data management, irrespective of the underlying storage systems. Multiple data container connections create a cohesive wide-area storage network, ensuring resilience using erasure coding policies. Furthermore, a load-balancing algorithm ensures equitable and efficient utilization of storage resources. We evaluate DynoStore using benchmarks and real-world case studies, including the management of medical and satellite data across geographically distributed environments. Our results demonstrate a 10% performance improvement compared to centralized cloud-hosted systems while maintaining competitive performance with state-of-the-art solutions such as Redis and IPFS. DynoStore also exhibits superior fault tolerance, withstanding more failures than traditional systems.
Problem
Research questions and friction points this paper is trying to address.
Manages data across heterogeneous storage systems
Provides unified coordination for disparate access protocols
Ensures resilience and load-balancing in wide-area networks
Innovation
Methods, ideas, or system contributions that make the work stand out.
Data containers for unified storage abstraction
Erasure coding for resilient wide-area network
Load-balancing for efficient resource utilization
🔎 Similar Papers
No similar papers found.
