SAGE: Signal-Amplified Guided Embeddings for LLM-based Vulnerability Detection

2026-04-20

AI Summary

This work addresses the "signal drowning" problem in large language models (LLMs) for vulnerability detection, where critical vulnerability signals are obscured by dominant functional semantics, thereby limiting detection performance. The authors propose SAGE, a novel framework that is the first to explicitly identify and mitigate this issue by leveraging task-conditioned sparse autoencoders and sparse manifold projection to actively recover and amplify weak internal vulnerability signals, achieving a 12.7× improvement in internal signal-to-noise ratio. SAGE establishes new state-of-the-art results across multiple benchmarks: a 7B-parameter model attains a 318% increase in Matthews Correlation Coefficient (MCC) on out-of-distribution data and consistently outperforms a 34B-parameter baseline across 13 programming languages, demonstrating substantially enhanced generalization and detection accuracy.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Software vulnerabilities are a primary threat to modern infrastructure. While static analysis and Graph Neural Networks have long served as the foundation for vulnerability detection, the emergence of Large Language Models (LLMs) has introduced a transformative paradigm driven by superior semantic reasoning and cross-environment generalization. However, in the context of LLM-based vulnerability detection, we identify a fundamental bottleneck in these models termed \textbf{Signal Submersion}: a state where features related to vulnerability are activated internally but numerically overwhelmed by dominant functional semantics. To address this, we propose \textbf{SAGE} (\textbf{S}ignal-\textbf{A}mplified \textbf{G}uided \textbf{E}mbeddings), a framework that shifts from passive signal submersion to active signal recovery. SAGE integrates task-conditional Sparse Autoencoders (SAEs) to isolate and amplify these faint vulnerability signals. Extensive evaluations on BigVul, PrimeVul, and PreciseBugs demonstrate that SAGE achieves state-of-the-art performance. Notably, SAGE mitigates Signal Submersion by increasing the internal Signal-to-Noise Ratio (SNR) by 12.7$\times$ via sparse manifold projection. This mechanistic intervention enables a 7B model to achieve up to 318\% Matthews Correlation Coefficient (MCC) gains on unseen distributions and a 319\% gain on classic datasets. By maintaining robust performance across 13 programming languages and outperforming 34B baselines, SAGE establishes a more efficient and scalable path to software security than simple parameter scaling.

Problem

• Signal Submersion
• Large Language Models
• Vulnerability Detection
• Software Security
• Semantic Reasoning

Innovation

• Signal Submersion
• Sparse Autoencoders
• Signal-to-Noise Ratio
• LLM-based Vulnerability Detection
• Guided Embeddings