Robust and Fine-Grained Detection of AI Generated Texts

2025-04-16

AI Summary

This paper addresses the challenges of generalization and fine-grained attribution in AI-generated text detection—specifically under cross-model, cross-lingual, short-text, human-AI collaborative, and adversarial perturbation settings—and proposes the first token-level robust detection framework tailored for human-AI co-authored texts. Methodologically, it integrates multilingual pretraining adaptation, cross-domain transfer learning, adversarial robust training, and fine-grained attribution analysis. Key contributions include: (1) the first systematic study of human-AI collaborative writing detection; (2) the release of a large-scale, multi-source co-authoring dataset covering 23 languages and 2.4 million samples; and (3) state-of-the-art performance under challenging conditions—including unknown generators, sub-50-token inputs, non-English texts, and adversarial attacks—validated across diverse domains and major LLMs.

Technology Category

• Natural Language Processing: Safety and Robustness
• Humans and AI: Human-AI Collaboration / Human-AI Teaming
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

An ideal detection system for machine generated content is supposed to work well on any generator as many more advanced LLMs come into existence day by day. Existing systems often struggle with accurately identifying AI-generated content over shorter texts. Further, not all texts might be entirely authored by a human or LLM, hence we focused more over partial cases i.e human-LLM co-authored texts. Our paper introduces a set of models built for the task of token classification which are trained on an extensive collection of human-machine co-authored texts, which performed well over texts of unseen domains, unseen generators, texts by non-native speakers and those with adversarial inputs. We also introduce a new dataset of over 2.4M such texts mostly co-authored by several popular proprietary LLMs over 23 languages. We also present findings of our models' performance over each texts of each domain and generator. Additional findings include comparison of performance against each adversarial method, length of input texts and characteristics of generated texts compared to the original human authored texts.

Problem

• Detect AI-generated texts robustly across advanced LLMs
• Improve accuracy in identifying short AI-generated texts
• Address human-LLM co-authored text detection challenges

Innovation

• Token classification models for AI text detection
• Extensive dataset of human-machine co-authored texts
• Robust performance across domains and generators