IM: Optimizing Byzantine Consensus for High-Performance Distributed Networks

2024-09-28

arXiv.org

AI Summary

To address the significant performance degradation of Byzantine Fault Tolerant (BFT) consensus under adversarial attacks and network instability, this paper proposes the Shared Memory Pool (SMP) protocol IM. Methodologically, IM introduces a novel micro-blockchain architecture synergized with lightweight erasure coding (EC), enabling three new guarantees: order preservation, bandwidth adaptivity, and overdistributed robustness. Crucially, IM is designed for seamless integration into existing BFT protocols—e.g., Fast-HotStuff—without requiring modifications to their core logic. Evaluation on a 256-node testbed demonstrates that, under a challenging setting with 100 active nodes (including 33 Byzantine nodes), IM achieves 9× higher throughput and 10× lower end-to-end latency compared to Stratus-FHS. Moreover, its performance advantage scales positively with the number of faulty nodes, confirming superior resilience in highly adversarial environments.

Technology Category

• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Byzantine Fault Tolerant (BFT) consensus, a crucial component of blockchains, has made significant advancements. However, the efficiency of existing protocols can still be damaged by certain attacks from faulty nodes and network instability. In this paper, we propose a novel Shared Mempool (SMP) protocol, namely IM, that enhances performance under these attacks. Technically, IM organizing microblocks into chains, combined with coding techniques, achieves totality and availability efficiently. IM can be easily integrated into a BFT protocol. We take Fast-HotStuff as an example and obtain the IM-FHS with guarantees of emph{order keeping}, emph{bandwidth adaptability} and emph{over-distribution resistance}. IM-FHS is conducted in a system with up to 256 nodes, and experimental results validate the efficiency of our approach. IM-FHS achieves higher throughput and smaller latency with faulty nodes than Stratus-FHS, the state-of-the-art protocol, and the throughput gain increases as the number of fault nodes. In a system with 100 nodes with 33 faulty nodes, IM-FHS achieves 9 times the throughput of Stratus-FHS while maintaining 1/10 the latency when dealing with maximum resilience against faulty nodes.

Problem

• Enhancing BFT consensus efficiency under network instability
• Mitigating over-distribution and unbalanced workload risks
• Improving throughput and latency with faulty nodes

Innovation

• Shared Mempool protocol with microblock chaining
• Coding techniques ensure totality and availability
• Integrates with BFT for order preservation