Clustering Unsupervised Representations as Defense against Poisoning Attacks on Speech Commands Classification System

2026-06-27

AI Summary

This work addresses the vulnerability of speech command classification systems to dirty-label poisoning attacks, wherein adversaries inject a trigger into samples from a source class and relabel them as a target class, thereby degrading model reliability. To counter this threat without requiring a clean validation set, the authors propose an unsupervised defense mechanism that integrates DINO-based self-supervised representation learning with clustering. Specifically, training samples are clustered using K-means followed by Linear Discriminant Analysis (LDA), and only those samples exhibiting the highest label consistency within each cluster are retained for model training. Experimental results demonstrate that, under a 10% poisoning rate in the source class, the attack success rate drops dramatically from 99.75% to 0.25%, with consistent robustness observed across diverse attack configurations.

Abstract

Poisoning attacks entail attackers intentionally tampering with training data. In this paper, we consider a dirty-label poisoning attack scenario on a speech commands classification system. The threat model assumes that certain utterances from one of the classes (source class) are poisoned by superimposing a trigger on it, and its label is changed to another class selected by the attacker (target class). We propose a filtering defense against such an attack. First, we use DIstillation with NO labels (DINO) to learn unsupervised representations for all the training examples. Next, we use K-means and LDA to cluster these representations. Finally, we keep the utterances with the most repeated label in their cluster for training and discard the rest. For a 10% poisoned source class, we demonstrate a drop in attack success rate from 99.75% to 0.25%. We test our defense against a variety of threat models, including different target and source classes, as well as trigger variations.

Problem

• poisoning attacks
• speech commands classification
• dirty-label
• adversarial defense
• data poisoning

Innovation

• unsupervised representation learning
• poisoning attack defense
• DINO
• clustering-based filtering
• speech command classification