Probing Stylistic Appropriation using Large Language Models: An Evaluation Framework for Copyright Infringement under EU Law

2026-06-30

AI Summary

Current copyright detection technologies primarily identify verbatim copying and fail to address “substantial similarity” as protected under EU copyright law—such as stylistic elements and narrative structures—creating a compliance gap. This work proposes PSALM, a novel framework that operationalizes the EU’s legal standard of substantial similarity into a computable, multidimensional assessment system. Leveraging an LLM-as-a-judge architecture, PSALM systematically evaluates infringement risk across ten dimensions, including computational overlap, stylistic resemblance, content alignment, and statutory exceptions. Experiments reveal that fine-tuned models often exhibit systematic stylistic appropriation beyond literal copying; while negative preference optimization reduces overall similarity, residual stylistic traces remain detectable, exposing limitations in current unlearning approaches.

Abstract

Large language models (LLM) trained on web-scale corpora generate output that may infringe copyright, yet existing technical safeguards focus narrowly on verbatim memorisation. EU copyright doctrine applies a broader standards: substantial similarity, which extends to stylistic choices, narrative structure, and creative elaboration. This mismatch between what current methods detect and what the law protects leaves a significant compliance gap. We introduce PSALM, an LLM-as-a-judge framework that operationalises EU copyright doctrine through ten evaluators assessing computational overlap, stylistic dimensions (writing style, narrative voice), content dimensions (character, plot, scene, world building), and statutory exceptions (parody, pastiche, quotation, scènes à faire). Applying PSALM to Llama~3.2 models fine-tuned on translated historical Dutch literary works, we find that: 1) instruction-tuned models exhibit non-trivial baseline stylistic similarity prior to corpus exposure; 2) fine-tuning induces systematic stylistic appropriation across all infringement-relevant dimensions, extending beyond verbatim memorisation to abstract narrative patterns; 3) Negative Preference Optimisation unlearning substantially reduces similarity but leaves detectable residual stylistic patterns. These findings indicate that safeguards targeting literal copying alone are insufficient to mitigate broader copyright risks. PSALM provides infrastructure for auditable, legally informed compliance evaluation, though the relationship between automated similarity scores and infringement determinations requires validation by legal experts. This work bridges qualitative legal standards and quantitative technical measurement, exposing fundamental tensions between generative AI and EU intellectual property law.

Problem

• copyright infringement
• stylistic appropriation
• substantial similarity
• large language models
• EU copyright law

Innovation

• stylistic appropriation
• copyright infringement
• LLM-as-a-judge
• substantial similarity
• EU copyright law