Detecting LLM-Generated Text with Performance Guarantees

2026-01-10

arXiv.org

AI Summary

The proliferation of highly realistic text generated by large language models has intensified risks related to misinformation and academic misconduct, underscoring the urgent need for reliable detection methods. This work proposes an online classifier that distinguishes human- from model-generated text without relying on watermarks or prior knowledge of the generative model, operating efficiently on CPU alone. By integrating statistical learning with computationally efficient feature modeling, the method introduces, for the first time, controllable statistical inference guarantees that rigorously bound Type I error while achieving high statistical power, superior classification accuracy, and strong computational efficiency. Empirical evaluations demonstrate that the proposed detector significantly outperforms existing approaches across multiple benchmarks.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Large language models (LLMs) such as GPT, Claude, Gemini, and Grok have been deeply integrated into our daily life. They now support a wide range of tasks -- from dialogue and email drafting to assisting with teaching and coding, serving as search engines, and much more. However, their ability to produce highly human-like text raises serious concerns, including the spread of fake news, the generation of misleading governmental reports, and academic misconduct. To address this practical problem, we train a classifier to determine whether a piece of text is authored by an LLM or a human. Our detector is deployed on an online CPU-based platform https://huggingface.co/spaces/stats-powered-ai/StatDetectLLM, and contains three novelties over existing detectors: (i) it does not rely on auxiliary information, such as watermarks or knowledge of the specific LLM used to generate the text; (ii) it more effectively distinguishes between human- and LLM-authored text; and (iii) it enables statistical inference, which is largely absent in the current literature. Empirically, our classifier achieves higher classification accuracy compared to existing detectors, while maintaining type-I error control, high statistical power, and computational efficiency.

Problem

• LLM-generated text
• fake news
• academic misconduct
• text detection
• human-like text

Innovation

• LLM-generated text detection
• statistical inference
• watermark-free detection
• type-I error control
• computational efficiency