Mitigating Package Hallucinations in Large Language Models via Model Editing

2026-07-02

AI Summary

This work addresses the critical issue of package hallucination—where large language models generate invalid or fabricated package names during software development—posing significant supply chain security risks. The authors propose BOUND, a novel framework that formalizes hallucination mitigation as an internal validity boundary editing problem within the model. BOUND employs a risk-aware module to identify critical regions and applies lightweight LoRA adapters for boundary-aware editing, effectively suppressing hallucinations while preserving the model’s capability to recommend valid packages. Experimental results demonstrate that BOUND reduces package-level hallucination rates by 79.9% under in-prompt settings and 65.4% under out-of-prompt settings in package recommendation tasks. Furthermore, the approach generalizes effectively to code generation and pip installation recommendation tasks, lowering hallucination rates by 12.8% and 34.0%, respectively.

Abstract

Large language models (LLMs) have demonstrated strong capabilities in software engineering tasks, such as code generation, library recommendation, and dependency configuration. However, recent studies show that LLMs may suffer from package hallucination, where they generate non-existent or invalid package names. These hallucinations can be exploited in software supply chain attacks, as attackers may register malicious packages under hallucinated names. Therefore, mitigating package hallucination is important for improving the reliability and security of LLM-assisted software development. In this paper, we introduce BOUND, a lightweight localized model editing framework for mitigating package hallucinations in LLMs. BOUND formulates package hallucination mitigation as a package-validity boundary editing problem, where the boundary refers to the model's ability to distinguish valid packages from hallucinated package names under a given task context. It first locates modules related to package hallucination through a risk-aware localization strategy, and then edits these modules with lightweight LoRA adapters using a boundary-aware objective that reinforces valid packages, suppresses hallucinated packages, and preserves locality behavior. Experimental results show that BOUND effectively reduces package hallucinations while preserving valid package recommendations. In the package recommendation task, BOUND reduces package-level hallucination rate (Package-HR) by 79.9% on edit prompts and by 65.4% on unseen prompts. The learned package-validity boundary further generalizes to other package-related tasks, reducing Package-HR by 12.8% in code generation and by 34.0% in pip install recommendation. These results show that BOUND refines the package-validity boundary of LLMs and improves the reliability of package-related outputs.

Problem

• package hallucination
• large language models
• software supply chain security
• model reliability

Innovation

• model editing
• package hallucination
• LoRA
• validity boundary
• software supply chain security