vMODB: Unifying event and data management for distributed asynchronous applications

2025-04-28

AI Summary

In event-driven architectures (EDA), asynchronous decoupling impedes cross-component transactional consistency: databases lack cross-service transaction awareness, and messaging systems cannot track application state, forcing developers to abandon ACID guarantees. This paper introduces the first distributed event streaming framework supporting end-to-end ACID semantics. Its core innovation is the Virtual Microservice (VMS) programming model, which explicitly captures cross-component transactions, data dependencies, and concurrency semantics. A unified execution engine—integrating event logging and state management—performs semantic parsing and distributed transaction scheduling. Evaluated on two benchmark workloads, the framework achieves up to 3× higher throughput than mainstream eventual-consistency approaches while strictly enforcing strong consistency.

Technology Category

• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Search and Optimization: Distributed Search
• Machine Learning: Distributed Machine Learning & Federated Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Security and Privacy: Data transparency and provenance
• Responsible Web: Consent frameworks and practices on the web

Abstract

Event-driven architecture (EDA) has emerged as a crucial architectural pattern for scalable cloud applications. However, its asynchronous and decoupled nature introduces challenges for meeting transactional requirements. Database systems, relegated to serving as storage engines for individual components, do not recognize transactions that span multiple components in EDAs. In contrast, messaging systems are unaware of the components' application states. Weaving such asynchronous and independent EDA components forces developers to relinquish transactional guarantees, resulting in data consistency issues. To address this challenge, we design vMODB, a distributed framework that enables the implementation of highly consistent and scalable cloud applications without compromising the envisioned benefits of EDA. We propose Virtual Micro Service (VMS), a novel programming model that provides familiar constructs to enable developers to specify the data model, constraints, and concurrency semantics of components, as well as transactions and data dependencies that span across components. vMODB leverages VMS semantics to enforce ACID properties by transparently unifying event logs and state management into a common event-driven execution framework. Our experiments using two benchmarks show that vMODB outperforms a widely adopted state-of-the-art competing framework that only offers eventual consistency by up to 3X. With its high performance, familiar programming constructs, and ACID properties, vMODB will significantly simplify the development of highly consistent and efficient EDAs.

Problem

• Addressing transactional challenges in event-driven architectures
• Unifying event logs and state management for consistency
• Enabling ACID properties in distributed asynchronous applications

Innovation

• Unifies event logs and state management transparently
• Introduces Virtual Micro Service programming model
• Enforces ACID properties in event-driven architectures