Assessing Contamination in Large Language Models: Introducing the LogProber method

2024-08-26

arXiv.org

AI Summary

This paper addresses the problem of overconfident evaluation of large language models (LLMs) due to training data contamination—particularly in short-text domains such as psychological scales. We propose LogProber, the first quantifiable contamination detection method tailored for short texts. It leverages sentence-level token log-probability modeling, sequence likelihood estimation, and statistical significance testing to efficiently identify data leakage in low-resource settings. Key contributions include: (i) the first empirical demonstration that instruction tuning and similar training paradigms can induce “stealth contamination”—where contamination exists despite negligible changes in token probabilities; and (ii) a systematic characterization of detection capability boundaries and failure conditions across mainstream training paradigms. Evaluated on diverse psychological questionnaire datasets, LogProber achieves high-precision contamination identification, offering a reproducible, interpretable diagnostic tool for fair LLM evaluation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

In machine learning, contamination refers to situations where testing data leak into the training set. The issue is particularly relevant for the evaluation of the performance of Large Language Models (LLMs), which are generally trained on gargantuan, and generally opaque, corpora of text scraped from the world wide web. Developing tools to detect contamination is therefore crucial to be able to fairly and properly track the evolution of the performance of LLMs. Most recent works in the field are not tailored to quantify contamination on short sequences of text like we find in psychology questionnaires. In the present paper we introduce LogProber, a novel, efficient, algorithm that we show able to detect contamination using token probability in given sentences. In the second part we investigate the limitations of the method and discuss how different training methods can contaminate models without leaving traces in the token probabilities.

Problem

• Detect contamination in LLM training data
• Differentiate confidence from contamination in responses
• Improve black-box contamination detection efficiency

Innovation

• Detects contamination via question familiarity
• Operates in black box LLM settings
• Compares favorably to concurrent approaches