Addressing The Devastating Effects Of Single-Task Data Poisoning In Exemplar-Free Continual Learning

2025-07-05

AI Summary

This work identifies and addresses a previously overlooked security threat in continual learning (CL)—single-task poisoning (STP)—where an attacker manipulates only the data of the current task, without access to historical or future task knowledge, to simultaneously degrade model stability (performance on past tasks) and plasticity (adaptability to new tasks). To this end, we propose the first threat model for sample-free CL under STP; design a lightweight poisoning detection method based on task-vector deviation; and develop a three-tier defense framework integrating detection, data purification, and retraining. Under standard image-based poisoning attacks, experiments show STP reduces average accuracy by up to 32.7%. Our framework effectively restores the stability–plasticity trade-off, boosting robustness across multiple CL benchmarks to near attack-free levels.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements

Abstract

Our research addresses the overlooked security concerns related to data poisoning in continual learning (CL). Data poisoning - the intentional manipulation of training data to affect the predictions of machine learning models - was recently shown to be a threat to CL training stability. While existing literature predominantly addresses scenario-dependent attacks, we propose to focus on a more simple and realistic single-task poison (STP) threats. In contrast to previously proposed poisoning settings, in STP adversaries lack knowledge and access to the model, as well as to both previous and future tasks. During an attack, they only have access to the current task within the data stream. Our study demonstrates that even within these stringent conditions, adversaries can compromise model performance using standard image corruptions. We show that STP attacks are able to strongly disrupt the whole continual training process: decreasing both the stability (its performance on past tasks) and plasticity (capacity to adapt to new tasks) of the algorithm. Finally, we propose a high-level defense framework for CL along with a poison task detection method based on task vectors. The code is available at https://github.com/stapaw/STP.git .

Problem

• Addressing data poisoning threats in continual learning systems
• Analyzing single-task poison attacks with limited adversary access
• Proposing defense framework against poisoning in continual learning

Innovation

• Focuses on single-task poison threats
• Uses standard image corruptions for attacks
• Proposes defense with task vector detection