Rethinking AI-Generated Text Detection: A Strong Baseline and the Distribution-Shift Problem That Remains

2026-07-03

AI Summary

Current methods for detecting AI-generated text suffer significant performance degradation under distribution shifts—such as changes in topic or generative model—and often misclassify human-written text as machine-generated with high confidence. This work proposes fully fine-tuned RoBERTa as a strong baseline, demonstrating that recent performance gains primarily stem from thorough fine-tuning rather than architectural complexity, and argues that robustness should be a central evaluation criterion. Building on this insight, the study introduces a domain adaptation approach combining LoRA-based K-shot first-order MAML with a sample-level confidence-weighted ensemble strategy. This method maintains strong in-distribution performance while substantially improving cross-domain detection robustness.

Abstract

Recent AI-generated text detection work often introduces a new benchmark together with a specialized detector tailored to it. We revisit this practice from a baseline-first perspective. Across several benchmarks, we show that a plain, fully fine-tuned RoBERTa matches or exceeds the specialized detectors those benchmarks are built around. This suggests that much of the recent architectural complexity is not what drives strong in-distribution detection. The remaining challenge is the distribution shift. The same strong baseline degrades sharply when the topic domain or generating model changes at test time, and simply adding more source data does not close the gap. We identify a key failure mode: under distribution shift, the detector can assign high-confidence machine labels to human-written text from unseen domains. We then study two lightweight domain adaptation methods to address this problem: $K$-shot adaptation with first-order MAML over LoRA adapters, and a per-sample confidence-weighted ensemble built on top of the adapted detector. Overall, our results suggest that progress in AI-generated text detection should be measured not only by in-distribution performance, but also by robustness under distribution shift.

Problem

• AI-generated text detection
• distribution shift
• domain adaptation
• out-of-distribution robustness

Innovation

• distribution shift
• AI-generated text detection
• RoBERTa baseline
• domain adaptation
• LoRA