I Can Find You in Seconds! Leveraging Large Language Models for Code Authorship Attribution

2025-01-14

AI Summary

Source code authorship attribution in cross-language, multi-author settings poses significant challenges for software forensics and code copyright protection. Method: This paper presents the first systematic evaluation of large language models (LLMs) for zero-shot and few-shot code author identification. We propose a tournament-style attribution framework to mitigate classification degradation under large author sets, integrate code-style–oriented prompt engineering, and conduct adversarial robustness evaluation to enhance generalizability and trustworthiness. Results: Our approach achieves a Matthews Correlation Coefficient (MCC) of 0.78 for zero-shot binary discrimination and 0.77 for few-shot multi-class attribution. On large-scale datasets—500-author C++ and 686-author Java—it attains 65.0% and 68.7% top-1 accuracy, respectively, using only one reference sample per author—substantially outperforming prior methods. This work establishes a scalable, robust paradigm for low-resource, cross-language code authorship analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Source code authorship attribution is important in software forensics, plagiarism detection, and protecting software patch integrity. Existing techniques often rely on supervised machine learning, which struggles with generalization across different programming languages and coding styles due to the need for large labeled datasets. Inspired by recent advances in natural language authorship analysis using large language models (LLMs), which have shown exceptional performance without task-specific tuning, this paper explores the use of LLMs for source code authorship attribution. We present a comprehensive study demonstrating that state-of-the-art LLMs can successfully attribute source code authorship across different languages. LLMs can determine whether two code snippets are written by the same author with zero-shot prompting, achieving a Matthews Correlation Coefficient (MCC) of 0.78, and can attribute code authorship from a small set of reference code snippets via few-shot learning, achieving MCC of 0.77. Additionally, LLMs show some adversarial robustness against misattribution attacks. Despite these capabilities, we found that naive prompting of LLMs does not scale well with a large number of authors due to input token limitations. To address this, we propose a tournament-style approach for large-scale attribution. Evaluating this approach on datasets of C++ (500 authors, 26,355 samples) and Java (686 authors, 55,267 samples) code from GitHub, we achieve classification accuracy of up to 65% for C++ and 68.7% for Java using only one reference per author. These results open new possibilities for applying LLMs to code authorship attribution in cybersecurity and software engineering.

Problem

• Software Forensics
• Authorship Attribution
• Multilingual Coding Styles

Innovation

• Large Language Models
• Authorship Attribution
• Forensic Analysis