Functional and Secure Code Generation with Task Vectors

2026-07-08

AI Summary

This work addresses the challenge of simultaneously ensuring functional correctness and security in code generation with large language models, a trade-off that existing approaches often handle through post-processing or disjoint evaluation. The authors propose SecVecCoder, the first method to introduce task vectors into code generation, achieving joint alignment of functionality and security by linearly modulating model weights—without requiring additional decoding strategies or post-hoc fixes. Evaluated on the CodeGuard+ benchmark, SecVecCoder significantly improves the rate of trustworthy code generation across six prominent code large language models, yielding gains of 2.1 to 36.0 percentage points. Notably, it achieves up to a 39.1-percentage-point improvement on previously unseen CWE categories, while incurring only a 0.6% increase in inference latency.

Abstract

Large language models (LLMs) are increasingly used for code generation, but they struggle to generate functional code free of security vulnerabilities. Prior work to improve the secure code generation abilities of such coding LLMs has largely focused on evaluating code functionality and security separately using different datasets, or focused on finding vulnerabilities post-generation. At the same time, the text-generation domain has seen significant work on alignment techniques, where models are tuned such that their outputs exhibit certain qualities (e.g., helpfulness, harmlessness). Of particular interest is task-vector arithmetic, where linear operations on LLM weights can be used to arbitrarily enhance alignment while incurring only minimal computational overhead. We develop a novel method, SecVecCoder, leveraging task vectors to produce trustworthy code that is simultaneously functional and secure without the need for post-generation adjustment. Across six coding LLMs from three families on the CodeGuard+ benchmark, SecVecCoder improves the rate of trustworthy code completions by 2.1-36.0 percentage points over the base model, with improvements on unseen CWE types reaching up to 39.1 percentage points. Since the effectiveness of the coding LLM relies only on changing the model weights, SecVecCoder requires no method-specific decoding and hence achieves a decoding latency within 0.6% of the base model's, on average.

Problem

• code generation
• functional code
• security vulnerabilities
• trustworthy code
• large language models

Innovation

• task vectors
• secure code generation
• functional correctness
• weight arithmetic
• LLM alignment