WeSCE: A Benchmark for Measuring Security Drift in LLM-Driven Code Editing

2026-08-15

AI Summary

This study addresses the challenge of quantifying security drift in LLM-based code editing under weak safety constraints by constructing a benchmark dataset comprising 400 real-world programs. We propose a continuous risk representation and heterogeneous vulnerability signal aggregation method, alongside a unified risk aggregation formula and multi-scale drift metrics spanning average to worst-case scenarios. This approach enables precise, multidimensional assessment of security risk variations during functional editing, effectively bridging the gap in measuring code editing security drift. Ultimately, this work provides both theoretical foundations and empirical tools for understanding and mitigating risks associated with large language model code generation, facilitating more robust safety assurance in automated programming tasks.

Abstract

In this work, we introduce WeSCE, a benchmark for quantifying security drift in code editing under weak-security constraints, where tasks specify only functional objectives without explicit security requirements. WeSCE consists of 400 executable programs derived from real-world code, covering feature addition, feature removal, bug fixing, and refactoring. To quantify security drift, we propose a continuous risk representation that aggregates heterogeneous vulnerability signals through a unified formulation, and define drift measures capturing changes in overall risk, worst-case severity, and vulnerability distribution under code transformations, providing a multi-scale view of security spanning average-case behavior to worst-case emphasis.

Problem

• Security Drift
• LLM-Driven Code Editing
• Weak-Security Constraints
• Benchmark

Innovation

• Security Drift
• Weak-Security Constraints
• Continuous Risk Representation
• Code Editing Benchmark
• Multi-scale Security Measurement