De-VertiFL: A Solution for Decentralized Vertical Federated Learning

2024-10-08

arXiv.org

AI Summary

To address privacy leakage and communication inefficiency in decentralized vertical federated learning (VFL) involving heterogeneous features and co-located entities across multiple parties, this paper proposes the first fully decentralized VFL framework. Methodologically, it introduces: (1) a decentralized network topology with distributed consensus-driven training coordination; (2) a secure hidden-layer output protection scheme combining secret sharing and partial homomorphic encryption, enabling privacy-preserving intermediate representation sharing and computational reuse; and (3) unified support for both image and tabular data in binary/multi-class classification tasks. Evaluated on multiple benchmark datasets, the framework achieves significantly higher F1-scores than existing VFL approaches, while guaranteeing strong privacy under the semi-honest security model and enabling high-accuracy, serverless fully distributed deployment.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Large Vision Models
• Search and Optimization: Distributed Search

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems

Abstract

Federated Learning (FL), introduced in 2016, was designed to enhance data privacy in collaborative model training environments. Among the FL paradigm, horizontal FL, where clients share the same set of features but different data samples, has been extensively studied in both centralized and decentralized settings. In contrast, Vertical Federated Learning (VFL), which is crucial in real-world decentralized scenarios where clients possess different, yet sensitive, data about the same entity, remains underexplored. Thus, this work introduces De-VertiFL, a novel solution for training models in a decentralized VFL setting. De-VertiFL contributes by introducing a new network architecture distribution, an innovative knowledge exchange scheme, and a distributed federated training process. Specifically, De-VertiFL enables the sharing of hidden layer outputs among federation clients, allowing participants to benefit from intermediate computations, thereby improving learning efficiency. De-VertiFL has been evaluated using a variety of well-known datasets, including both image and tabular data, across binary and multiclass classification tasks. The results demonstrate that De-VertiFL generally surpasses state-of-the-art methods in F1-score performance, while maintaining a decentralized and privacy-preserving framework.

Problem

• Decentralized Vertical Federated Learning
• Privacy-preserving model training
• Efficient knowledge exchange scheme

Innovation

• Decentralized network architecture distribution
• Innovative knowledge exchange scheme
• Distributed federated training process