Membership Inference Attacks on Tokenizers of Large Language Models

2025-10-07

AI Summary

This work identifies, for the first time, tokenizers—core components of large language models (LLMs)—as a novel and potent attack surface for membership inference attacks (MIAs), addressing key challenges of label noise, distributional shift, and scale mismatch that hinder traditional MIAs on LLMs. Method: Leveraging the high overlap between tokenizer training data and LLM pretraining corpora, and exploiting tokenizers’ lightweight, deterministic nature, the authors propose a novel MIA framework grounded in statistical features of tokenization behavior. They empirically evaluate five distinct attack strategies across mainstream tokenizers (e.g., BPE, SentencePiece) on large-scale web text. Contribution/Results: The study demonstrates significant membership information leakage in widely used tokenizers; further, it introduces an adaptive defense mechanism that substantially reduces attack success rates with negligible impact on tokenization accuracy or efficiency. This work establishes the first systematic investigation into tokenizer privacy risks, bridging a critical gap in LLM privacy research and advancing holistic, stack-wide privacy protection for foundation models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Security and Privacy: Large-scale security measurements

Abstract

Membership inference attacks (MIAs) are widely used to assess the privacy risks associated with machine learning models. However, when these attacks are applied to pre-trained large language models (LLMs), they encounter significant challenges, including mislabeled samples, distribution shifts, and discrepancies in model size between experimental and real-world settings. To address these limitations, we introduce tokenizers as a new attack vector for membership inference. Specifically, a tokenizer converts raw text into tokens for LLMs. Unlike full models, tokenizers can be efficiently trained from scratch, thereby avoiding the aforementioned challenges. In addition, the tokenizer's training data is typically representative of the data used to pre-train LLMs. Despite these advantages, the potential of tokenizers as an attack vector remains unexplored. To this end, we present the first study on membership leakage through tokenizers and explore five attack methods to infer dataset membership. Extensive experiments on millions of Internet samples reveal the vulnerabilities in the tokenizers of state-of-the-art LLMs. To mitigate this emerging risk, we further propose an adaptive defense. Our findings highlight tokenizers as an overlooked yet critical privacy threat, underscoring the urgent need for privacy-preserving mechanisms specifically designed for them.

Problem

• Investigating tokenizers as a new vector for membership inference attacks on LLMs
• Addressing limitations of traditional MIAs by exploiting tokenizer training data
• Proposing defenses against privacy risks from tokenizer-based membership leakage

Innovation

• Uses tokenizers as new attack vector for membership inference
• Proposes five attack methods to infer dataset membership
• Introduces adaptive defense to mitigate tokenizer privacy risks