🤖 AI Summary
Traditional vertical federated learning (VFL) is constrained by the sample-overlap assumption and incurs high real-time inference overhead, rendering it ill-suited for online advertising. To address these limitations, this paper proposes Semi-VFL—a novel vertical semi-federated learning paradigm that eliminates the requirement for sample overlap and enables modeling over the full sample space. We design a Joint Privileged Learning (JPL) framework coupled with cross-party representation distillation to jointly train on both overlapping and non-overlapping data. Furthermore, we introduce a lightweight single-party student model and cross-party feature correlation modeling to balance prediction accuracy, inference efficiency, and feasibility of localized deployment. Evaluated on real-world advertising datasets, Semi-VFL consistently outperforms state-of-the-art baselines, achieving significant improvements in AUC, queries-per-second (QPS), and deployment flexibility.
📝 Abstract
The traditional vertical federated learning schema suffers from two main issues: 1) restricted applicable scope to overlapped samples and 2) high system challenge of real-time federated serving, which limits its application to advertising systems. To this end, we advocate a new learning setting Semi-VFL (Vertical Semi-Federated Learning) to tackle these challenge. Semi-VFL is proposed to achieve a practical industry application fashion for VFL, by learning a federation-aware local model which performs better than single-party models and meanwhile maintain the convenience of local-serving. For this purpose, we propose the carefully designed Joint Privileged Learning framework (JPL) to i) alleviate the absence of the passive party's feature and ii) adapt to the whole sample space. Specifically, we build an inference-efficient single-party student model applicable to the whole sample space and meanwhile maintain the advantage of the federated feature extension. New representation distillation methods are designed to extract cross-party feature correlations for both the overlapped and non-overlapped data. We conducted extensive experiments on real-world advertising datasets. The results show that our method achieves the best performance over baseline methods and validate its superiority in the Semi-VFL setting.