Kill two birds with one stone: generalized and robust AI-generated text detection via dynamic perturbations

2025-04-22

North American Chapter of the Association for Computational Linguistics

AI Summary

AI-generated text (AIGT) detection faces a fundamental trade-off between generalization and robustness, hindering simultaneous optimization of both properties. Method: This paper proposes DP-Net, a reinforcement learning–based dynamic perturbation framework that explicitly models robustness as domain shift and introduces a unified dynamic perturbation mechanism to jointly optimize generalization and robustness. It comprises a customized reward function and action space, cross-domain generalization training, and adversarial robustness evaluation. Contribution/Results: DP-Net is the first to theoretically characterize the intrinsic coupling between generalization and robustness in AIGT detection. Empirically, it achieves state-of-the-art generalization performance across three cross-domain scenarios and attains optimal robustness against two prevalent classes of textual adversarial attacks. The implementation is publicly available.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

The growing popularity of large language models has raised concerns regarding the potential to misuse AI-generated text (AIGT). It becomes increasingly critical to establish an excellent AIGT detection method with high generalization and robustness. However, existing methods either focus on model generalization or concentrate on robustness. The unified mechanism, to simultaneously address the challenges of generalization and robustness, is less explored. In this paper, we argue that robustness can be view as a specific form of domain shift, and empirically reveal an intrinsic mechanism for model generalization of AIGT detection task. Then, we proposed a novel AIGT detection method (DP-Net) via dynamic perturbations introduced by a reinforcement learning with elaborated reward and action. Experimentally, extensive results show that the proposed DP-Net significantly outperforms some state-of-the-art AIGT detection methods for generalization capacity in three cross-domain scenarios. Meanwhile, the DP-Net achieves best robustness under two text adversarial attacks. The code is publicly available at https://github.com/CAU-ISS-Lab/AIGT-Detection-Evade-Detection/tree/main/DP-Net.

Problem

• Detect AI-generated text with high generalization and robustness
• Unify mechanisms for generalization and robustness in detection
• Improve detection under adversarial attacks and cross-domain scenarios

Innovation

• Dynamic perturbations for robust detection
• Reinforcement learning with reward and action
• Unified generalization and robustness mechanism