Probe-Free Low-Rank Activation Intervention

2025-02-06

AI Summary

Large language models (LLMs) frequently generate hallucinated or harmful content. Existing activation-based intervention methods rely on pretrained classifier probes to detect undesirable outputs, limiting generalizability and deployment efficiency. This paper proposes FLORAIN—the first probe-free activation intervention framework. It applies sample-adaptive, low-rank nonlinear corrections to hidden states within attention layers, optimizing toward an ideal semantic manifold via differentiable projection—enabling unsupervised, real-time intervention. Its core innovation lies in eliminating explicit detectors and instead modeling intervention as implicit semantic calibration under manifold constraints. Experiments demonstrate that FLORAIN significantly improves generation truthfulness and safety across multiple base models, consistently outperforming state-of-the-art intervention baselines on text generation and multiple-choice reasoning tasks. Moreover, it exhibits strong model-agnosticity and lightweight deployment characteristics.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Language models (LMs) can produce texts that appear accurate and coherent but contain untruthful or toxic content. Inference-time interventions that edit the hidden activations have shown promising results in steering the LMs towards desirable generations. Existing activation intervention methods often comprise an activation probe to detect undesirable generation, triggering the activation modification to steer subsequent generation. This paper proposes a probe-free intervention method FLORAIN for all attention heads in a specific activation layer. It eliminates the need to train classifiers for probing purposes. The intervention function is parametrized by a sample-wise nonlinear low-rank mapping, which is trained by minimizing the distance between the modified activations and their projection onto the manifold of desirable content. Under specific constructions of the manifold and projection distance, we show that the intervention strategy can be computed efficiently by solving a smooth optimization problem. The empirical results, benchmarked on multiple base models, demonstrate that FLORAIN consistently outperforms several baseline methods in enhancing model truthfulness and quality across generation and multiple-choice tasks.

Problem

• Detects untruthful or toxic content
• Proposes probe-free intervention method
• Enhances model truthfulness and quality

Innovation

• Probe-free intervention method
• Low-rank activation mapping
• Manifold projection optimization