Curvature-Guided Module Localization for Low-Rank Detoxification of Backdoored Large Language Models

2026-06-29

AI Summary

This work addresses the vulnerability of large language models to backdoor attacks, wherein malicious behaviors are triggered under specific conditions. To efficiently "detoxify" compromised models without retraining, the authors propose a mechanism-guided weight-space repair framework that formulates backdoor removal as a localized structural correction task. By integrating activation patching with Fisher information matrix and K-FAC curvature analysis, the method precisely identifies and applies low-rank updates to the critical modules responsible for propagating trigger-induced behaviors. Experiments on Llama-3.2-1B-Instruct demonstrate that the approach significantly suppresses malicious responses across diverse trigger placements while preserving the model’s benign capabilities, achieving a strong balance between repair accuracy and computational efficiency.

Abstract

Backdoor attacks pose a serious threat to large language models (LLMs) by causing otherwise benign systems to produce attacker-specified malicious behavior when a hidden trigger is present. In this work, we study post hoc detoxification of backdoored LLMs in a practical setting where the defender has access to the poisoned model but does not wish to retrain the full network from scratch. We propose a mechanistically guided weight-space repair framework that first localizes modules involved in propagating trigger-induced behavior using activation patching and Fisher/K-FAC curvature analysis, and then applies targeted low-rank repair to only the most influential modules. We evaluate the method on poisoned variants of \texttt{Llama-3.2-1B-Instruct} with triggers inserted at the beginning, middle, and end of otherwise benign prompts. Results show that the proposed approach substantially suppresses trigger-conditioned malicious responses while preserving benign model behavior. These findings suggest that backdoor removal in LLMs can be formulated as a localized structural repair problem rather than only a broad behavioral alignment problem.

Problem

• backdoor attacks
• large language models
• detoxification
• module localization
• low-rank repair

Innovation

• curvature-guided localization
• low-rank repair
• backdoor detoxification
• activation patching
• module-level intervention