PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture

2025-10-14

AI Summary

To address low training efficiency and poor resource utilization in vertical federated learning (VFL) caused by synchronous dependencies and system/data heterogeneity, this paper proposes a novel two-party asynchronous split learning paradigm based on a publish/subscribe architecture. Our method introduces a hierarchical asynchronous communication mechanism that synergistically integrates the decoupling property of publish/subscribe with the data-parallel capability of parameter servers. It further incorporates a system-characteristic-aware hyperparameter optimization strategy to mitigate training imbalance and natively supports privacy-preserving protocols such as differential privacy. Extensive experiments across five benchmark datasets demonstrate that our approach achieves 2–7× faster training compared to state-of-the-art methods, attains up to 91.07% computational resource utilization, and preserves model accuracy without degradation.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

With the rapid advancement of the digital economy, data collaboration between organizations has become a well-established business model, driving the growth of various industries. However, privacy concerns make direct data sharing impractical. To address this, Two-Party Split Learning (a.k.a. Vertical Federated Learning (VFL)) has emerged as a promising solution for secure collaborative learning. Despite its advantages, this architecture still suffers from low computational resource utilization and training efficiency. Specifically, its synchronous dependency design increases training latency, while resource and data heterogeneity among participants further hinder efficient computation. To overcome these challenges, we propose PubSub-VFL, a novel VFL paradigm with a Publisher/Subscriber architecture optimized for two-party collaborative learning with high computational efficiency. PubSub-VFL leverages the decoupling capabilities of the Pub/Sub architecture and the data parallelism of the parameter server architecture to design a hierarchical asynchronous mechanism, reducing training latency and improving system efficiency. Additionally, to mitigate the training imbalance caused by resource and data heterogeneity, we formalize an optimization problem based on participants' system profiles, enabling the selection of optimal hyperparameters while preserving privacy. We conduct a theoretical analysis to demonstrate that PubSub-VFL achieves stable convergence and is compatible with security protocols such as differential privacy. Extensive case studies on five benchmark datasets further validate its effectiveness, showing that, compared to state-of-the-art baselines, PubSub-VFL not only accelerates training by $2 sim 7 imes$ without compromising accuracy, but also achieves a computational resource utilization rate of up to 91.07%.

Problem

• Addresses low resource utilization in two-party split learning
• Reduces training latency through hierarchical asynchronous mechanisms
• Mitigates performance imbalance from resource and data heterogeneity

Innovation

• Uses Publisher/Subscriber architecture for efficient two-party learning
• Implements hierarchical asynchronous mechanism to reduce training latency
• Optimizes hyperparameters based on system profiles for imbalance mitigation