Unlearning Clients, Features and Samples in Vertical Federated Learning

2025-01-23

AI Summary

This work addresses the regulatory requirement for data deletion in vertical federated learning (VFL), systematically tackling efficient model unlearning across three dimensions: clients, features, and samples—the first such effort in VFL. We propose two communication-free unlearning methods: VFU-KD, based on knowledge distillation, and VFU-GA, based on gradient ascent—both eliminating the need for interaction between active and passive parties during the unlearning phase. To enable auditable evaluation, we adopt membership inference attacks (MIAs) as a practical, proxy metric for unlearning effectiveness. Extensive experiments on six tabular and two image datasets demonstrate that our methods achieve utility comparable to or better than from-scratch retraining and the R2S baseline, improving accuracy by 0–2% in most scenarios, with only marginal degradation (1–5%) in a few cases—striking a strong balance between thorough unlearning and model utility.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Federated Learning (FL) has emerged as a prominent distributed learning paradigm. Within the scope of privacy preservation, information privacy regulations such as GDPR entitle users to request the removal (or unlearning) of their contribution from a service that is hosting the model. For this purpose, a server hosting an ML model must be able to unlearn certain information in cases such as copyright infringement or security issues that can make the model vulnerable or impact the performance of a service based on that model. While most unlearning approaches in FL focus on Horizontal FL (HFL), where clients share the feature space and the global model, Vertical FL (VFL) has received less attention from the research community. VFL involves clients (passive parties) sharing the sample space among them while not having access to the labels. In this paper, we explore unlearning in VFL from three perspectives: unlearning clients, unlearning features, and unlearning samples. To unlearn clients and features we introduce VFU-KD which is based on knowledge distillation (KD) while to unlearn samples, VFU-GA is introduced which is based on gradient ascent. To provide evidence of approximate unlearning, we utilize Membership Inference Attack (MIA) to audit the effectiveness of our unlearning approach. Our experiments across six tabular datasets and two image datasets demonstrate that VFU-KD and VFU-GA achieve performance comparable to or better than both retraining from scratch and the benchmark R2S method in many cases, with improvements of $(0-2%)$. In the remaining cases, utility scores remain comparable, with a modest utility loss ranging from $1-5%$. Unlike existing methods, VFU-KD and VFU-GA require no communication between active and passive parties during unlearning. However, they do require the active party to store the previously communicated embeddings.

Problem

• Vertical Federated Learning
• Knowledge Forgetting
• Model Performance Preservation

Innovation

• Vertical Federated Learning
• Knowledge Distillation
• Gradient Ascent