Detection of LLM-assisted Code Plagiarism Using k-gram Software Birthmarks

2026-07-03

AI Summary

This study addresses the challenge of detecting large language model (LLM)-generated code that evades conventional plagiarism detection through semantics-preserving rewrites. It presents the first systematic evaluation of Java bytecode-based k-gram software watermarking (with k ranging from 1 to 6) in the context of LLM-generated code. The approach integrates five similarity metrics—cosine similarity, Dice coefficient, Jaccard index, Simpson index, and edit distance–based similarity—and is evaluated on code produced by three prominent LLMs. Results demonstrate that the proposed watermarking technique effectively identifies LLM-assisted plagiarism, with code generated by domain-specialized models (e.g., ChatGPT-5.1-Codex-Mini) exhibiting greater stealthiness, thereby confirming that model specialization enhances the concealment of plagiarized content.

Abstract

Large language models (LLMs) have significantly lowered the technical barrier to software plagiarism. By transforming existing source code while preserving its functionality, modern LLMs can generate semantically identical program that may evade traditional plagiarism detection techniques. Among such attacks, code paraphrasing modifies the syntax and structure of a program while preserving its behavior. This paper investigates whether software birthmarks can detect such LLM-assisted plagiarism. As a starting point, we employ k-gram software birthmarks based on unique k-grams of Java opcodes, with k ranging from 1 to 6. We employ three contemporary LLMs: ChatGPT-5.1-Codex-Mini, DeepSeek-V4-Flash, and Claude-Haiku-4.5. The dataset consists of individually compilable source files extracted from actively maintained BSD-2-Clause licensed Java projects. We further compare five similarity measures for birthmark matching: cosine similarity, Dice index, Jaccard coefficient, Simpson index, and edit-distance-based similarity. The results demonstrate that k-gram software birthmarks remain effective for detecting LLM-assisted plagiarism. Among the evaluated models, ChatGPT-5.1-Codex-Mini generated the most difficult-to-detect clones. Furthermore, the findings confirm the higher performance of coding-oriented models for plagiarism task.

Problem

• LLM-assisted plagiarism
• code paraphrasing
• software birthmarks
• plagiarism detection
• k-gram

Innovation

• k-gram software birthmarks
• LLM-assisted code plagiarism
• opcode-based detection
• code paraphrasing
• similarity measures