Interpretable Predictability-Based AI Text Detection: A Replication Study

2026-03-16

AI Summary

This work addresses the challenge of authorship attribution for machine-generated text by proposing a unified cross-lingual detection framework to enhance the reproducibility and generalizability of AI-generated text detection. The approach leverages mDeBERTa-v3-base as the contextual representation model, incorporates 26 document-level stylometric features, and replaces the original GPT-2 with Qwen and mGPT for computing probability-based features. Instead of language-specific models, a unified multilingual configuration is adopted, and SHAP analysis is integrated to improve decision interpretability. Experimental results demonstrate that the proposed method outperforms baseline models in both English and Spanish subtasks, confirming the effectiveness of the newly introduced features and the viability of the cross-lingual strategy.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

This paper replicates and extends the system used in the AuTexTification 2023 shared task for authorship attribution of machine-generated texts. First, we tried to reproduce the original results. Exact replication was not possible because of differences in data splits, model availability, and implementation details. Next, we tested newer multilingual language models and added 26 document-level stylometric features. We also applied SHAP analysis to examine which features influence the model's decisions. We replaced the original GPT-2 models with newer generative models such as Qwen and mGPT for computing probabilistic features. For contextual representations, we used mDeBERTa-v3-base and applied the same configuration to both English and Spanish. This allowed us to use one shared configuration for Subtask 1 and Subtask 2. Our experiments show that the additional stylometric features improve performance in both tasks and both languages. The multilingual configuration achieves the results that are comparable to or better than language-specific models. The study also shows that clear documentation is important for reliable replication and fair comparison of systems.

Problem

• AI text detection
• authorship attribution
• machine-generated text
• multilingual models
• replication study

Innovation

• stylometric features
• multilingual language models
• SHAP analysis
• AI text detection
• model interpretability