TracE2E: Easily Deployable Middleware for Decentralized Data Traceability

2025-10-09

AI Summary

In multi-node systems, ensuring data traceability, interpretability, and regulatory compliance simultaneously remains challenging, while retrofitting existing applications for these purposes incurs high engineering costs. Method: This paper proposes TracE2E—a lightweight, Rust-based I/O middleware that enables decentralized data lineage capture without modifying application code, achieved via standard-library I/O abstraction and system-call interception. It ensures cross-node lineage integrity through a distributed log consensus protocol and supports dynamic, lineage-driven data protection policy enforcement. Contribution/Results: TracE2E is the first middleware to unify non-intrusive provenance collection and policy execution within a single architecture. Evaluation shows low deployment overhead, broad compatibility with legacy applications, and substantial reduction in compliance auditing complexity—enabling rapid GDPR and similar regulatory adherence for existing systems.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

This paper presents TracE2E, a middleware written in Rust, that can provide both data explainability and compliance across multiple nodes. By mediating inputs and outputs of processes, TracE2E records provenance information and enforces data-protection policies (e.g., confidentiality, integrity) that depend on the recorded provenance. Unlike existing approaches that necessitate substantial application modifications, TracE2E is designed for easy integration into existing and future applications through a wrapper of the Rust standard library's IO module. We describe how TracE2E consistently records provenance information across nodes, and we demonstrate how the compliance layer of TracE2E can accommodate the enforcement of multiple policies.

Problem

• Provides decentralized data traceability across multiple nodes
• Enforces data protection policies using recorded provenance
• Enables easy integration without major application modifications

Innovation

• Middleware provides data explainability and compliance
• Records provenance by mediating process inputs and outputs
• Easy integration via wrapper of Rust standard library