Are Classical Clone Detectors Good Enough For the AI Era?

2025-09-30

AI Summary

This study investigates the effectiveness challenges of traditional code clone detection (CCD) tools when applied to AI-generated code. Motivated by the systematic syntactic patterns and semantic biases inherent in GPT-3–generated code, we conduct the first systematic evaluation of nine classical CCD tools across three benchmarks—GPTCloneBench, BigCloneBench, and SemanticCloneBench—and contrast their performance against manually written code. Our method includes controlled experiments with standardized preprocessing and quantitative analysis of accuracy, scalability, and execution efficiency. Results show that normalization-aware preprocessing is critical for improving detection precision; tools with robust normalization capabilities (e.g., Deckard, NICAD) maintain high accuracy and stability on AI-generated clones. We further provide empirical insights into tool scalability and runtime overhead, offering evidence-based guidance for CCD tool selection and adaptation in the AI era. This work establishes a methodological foundation and practical benchmarks for advancing CCD research in the context of large language model–generated code.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Other Foundations of Human Computation & AI
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The increasing adoption of AI-generated code has reshaped modern software development, introducing syntactic and semantic variations in cloned code. Unlike traditional human-written clones, AI-generated clones exhibit systematic syntactic patterns and semantic differences learned from large-scale training data. This shift presents new challenges for classical code clone detection (CCD) tools, which have historically been validated primarily on human-authored codebases and optimized to detect syntactic (Type 1-3) and limited semantic clones. Given that AI-generated code can produce both syntactic and complex semantic clones, it is essential to evaluate the effectiveness of classical CCD tools within this new paradigm. In this paper, we systematically evaluate nine widely used CCD tools using GPTCloneBench, a benchmark containing GPT-3-generated clones. To contextualize and validate our results, we further test these detectors on established human-authored benchmarks, BigCloneBench and SemanticCloneBench, to measure differences in performance between traditional and AI-generated clones. Our analysis demonstrates that classical CCD tools, particularly those enhanced by effective normalization techniques, retain considerable effectiveness against AI-generated clones, while some exhibit notable performance variation compared to traditional benchmarks. This paper contributes by (1) evaluating classical CCD tools against AI-generated clones, providing critical insights into their current strengths and limitations; (2) highlighting the role of normalization techniques in improving detection accuracy; and (3) delivering detailed scalability and execution-time analyses to support practical CCD tool selection.

Problem

• Evaluating classical clone detectors on AI-generated code variations
• Assessing performance differences between human-written and AI clones
• Analyzing effectiveness of normalization techniques for AI clones

Innovation

• Systematically evaluates nine classical clone detectors
• Uses GPTCloneBench benchmark for AI-generated code clones
• Highlights normalization techniques improving detection accuracy