Feature-Augmented Transformers for Robust AI-Text Detection Across Domains and Generators

2026-05-05

AI Summary

This work addresses the challenge of distribution shift faced by AI-generated text detectors in cross-domain and multi-generator settings by proposing an attention-based linguistic feature augmentation approach. Built upon the DeBERTa-v3-base backbone, the method integrates readability and lexical features to enhance model generalization under a fixed decision threshold protocol. Experimental results demonstrate that the proposed approach achieves a balanced accuracy of 85.9% on the M4 benchmark, outperforming strong zero-shot baselines by up to 7.22 percentage points. Robustness and stability are further validated across multiple datasets—including HC3 PLUS, M4, and AI-Text-Detection-Pile—as well as various ablation studies, which also reveal an asymmetry in detection errors.

Abstract

AI-generated text is nowadays produced at scale across domains and heterogeneous generation pipelines, making robustness to distribution shift a central requirement for supervised binary detectors. We train transformer-based detectors on HC3 PLUS and calibrate a single decision threshold by maximising balanced accuracy on held-out validation; this threshold is then kept fixed for all downstream test distributions, revealing domain- and generator-dependent error asymmetries under shift. We evaluate in-domain on HC3 PLUS, under cross-dataset transfer to the multi-domain, multi-generator M4 benchmark, and on the external AI-Text-Detection-Pile. Although base models achieve near-ceiling in-domain performance (up to 99.5% balanced accuracy), performance under shift is brittle and strongly model-dependent. Feature augmentation via attention-based linguistic feature fusion improves transfer, with our best model (DeBERTa-v3-base+FeatAttn) achieving 85.9% balanced accuracy on M4. Multi-seed experiments confirm high stability. Under the same fixed-threshold protocol, our model outperforms strong zero-shot baselines by up to +7.22 points. Category-level ablations further show that readability and vocabulary features contribute most to robustness under shift. Overall, these results demonstrate that feature augmentation and a modern DeBERTa backbone significantly outperform earlier BERT/RoBERTa models, while the fixed-threshold protocol provides a more realistic and informative assessment of practical detector robustness.

Problem

• AI-text detection
• distribution shift
• domain generalization
• robustness
• text generation

Innovation

• feature augmentation
• distribution shift
• AI-text detection
• fixed-threshold evaluation
• linguistic features