Institution profile

Technische Universität Clausthal

Academic institutioneurope · de
Official website
Research library65linked papers
Opportunities0open roles
Selected work

Representative Papers

Persistent Human Feedback, LLMs, and Static Analyzers for Secure Code Generation and Vulnerability Detection

Feb 05, 2026

This study addresses the significant inaccuracies of existing static analysis tools in evaluating the security of code generated by large language models (LLMs), which often fail to reliably identify genuine vulnerabilities. The work presents the first systematic investigation of this issue and introduces a human-verified benchmark dataset of secure and vulnerable code samples to assess the performance of widely used tools such as CodeQL and Semgrep, revealing sample-level accuracy rates of only 61% and 65%, respectively. To mitigate these limitations, the authors propose a dynamic retrieval-augmented generation (RAG) framework that integrates continuous human feedback, enabling persistent knowledge reuse and iterative improvement of security capabilities. Experimental results demonstrate that this approach substantially enhances the accuracy and reliability of LLMs in both secure code generation and vulnerability detection.

1 citationsRead paper

Can Developers rely on LLMs for Secure IaC Development?

Feb 03, 2026

This study addresses the limited reliability of current large language models (LLMs) in secure Infrastructure-as-Code (IaC) development, particularly their inability to effectively detect security smells and generate secure code. It presents the first systematic evaluation of GPT-4o and Gemini 2.0 Flash across real-world and synthetic IaC scenarios, employing both general and guided prompt engineering strategies, security smell analysis, and manual assessment. Results demonstrate that guided prompting substantially improves detection performance, identifying 78% and 67% of security smells in Stack Overflow and GitHub datasets, respectively. However, only 7%–17% of the generated code adheres to established security standards, underscoring the current inadequacy of LLMs for dependable secure IaC development. This work establishes an empirical benchmark and offers actionable guidance for prompt design in IaC security automation.

1 citationsRead paper

From Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models

Jan 30, 2026

This study addresses a critical yet overlooked issue in the evaluation of AI-driven keyphrase extraction models: severe performance overestimation due to data leakage between training and test sets caused by duplicated samples in benchmark datasets. For the first time, this work systematically quantifies and reveals the misleading impact of such leakage on model generalization assessments. Through large-scale duplicate detection, cross-set similarity analysis, and controlled experiments—augmented by code fingerprinting and semantic comparison to identify leaked instances—the authors propose a rigorous de-duplicated evaluation protocol. Re-evaluation under this protocol demonstrates that state-of-the-art models suffer substantial F1 score drops (exceeding 30% in some cases) once data leakage is eliminated, exposing significant risks in real-world deployment and establishing a more reliable, unbiased benchmark for future research in keyphrase extraction.

1 citationsRead paper

Evaluating Large Language Models for Security Bug Report Prediction

Jan 30, 2026

This study addresses the challenge of accurately identifying security bug reports (SBRs) at an early stage to facilitate timely remediation. It systematically evaluates the performance trade-offs between prompt-engineered closed-source large language models and fine-tuned open-source counterparts for SBR prediction. Experiments across multiple real-world datasets reveal that prompt-based models achieve a recall of 74% and a G-measure of 77%, but suffer from low precision (22%). In contrast, fine-tuned models attain substantially higher precision (75%) and up to 50× faster inference, albeit with a lower G-measure of 51%. This work is the first to uncover the complementary strengths of these two approaches—highlighting their distinct trade-offs in sensitivity, precision, and efficiency—and provides practical guidance for selecting appropriate strategies based on deployment contexts.

1 citationsRead paper
Recent publications

Latest Papers