Semantic Code Clone Detection: Are We There Yet?

2026-06-23

AI Summary

Existing semantic code clone detection methods perform well on standard benchmarks, yet it remains unclear whether they genuinely understand program semantic equivalence, particularly due to the lack of systematic evaluation of their out-of-distribution generalization. This work proposes an operator framework based on Type-2 and Type-3 clone transformations to generate Type-4 clone pairs that are semantically equivalent but exhibit distributional shifts. We conduct the first systematic generalization assessment of 11 state-of-the-art detectors on BigCloneBench, covering three technical paradigms: token-based, abstract syntax tree–based, and graph-based approaches. Results show a significant performance drop across all methods under distribution shift, revealing their fundamental reliance on lexical or structural shortcuts rather than deep semantic understanding.

Abstract

Code clone detection has been extensively studied for decades, and recent approaches have begun reporting remarkably high performance for semantic (Type-4) clones on benchmark datasets. However, it remains unclear whether these results reflect a genuine ability to capture semantic equivalence between programs, or simply an ability to exploit dataset-specific patterns. In this paper, we present the first systematic empirical study investigating the generalizability of state-of-the-art (SOTA) semantic code clone detectors beyond benchmark evaluation settings. Inspired by the inherent inclusion relationship among clone types, we propose a clone operator framework consisting of eight transformation operators derived from Type-2 and Type-3 clone variations. Using these operators, we construct distribution-shifted yet semantically equivalent Type-4 clone instances and evaluate 11 representative detectors spanning token-based, tree-based, and graph-based paradigms on the real-world BigCloneBench dataset. Our results reveal substantial performance degradation across all evaluated approaches, despite their strong benchmark performance. Further analyses show that existing detectors heavily rely on shortcut learning based on lexical and structural cues rather than robust semantic understanding. Our findings suggest that current SOTA semantic code clone detectors exhibit limited generalizability in real-world scenarios, highlighting important avenues for future research.

Problem

• semantic code clone detection
• generalizability
• distribution shift
• semantic equivalence
• benchmark evaluation

Innovation

• semantic code clone detection
• generalizability
• clone operator framework
• distribution shift
• shortcut learning