How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?

2025-02-20

AI Summary

This study investigates the capacity limits and generalization trade-offs inherent in knowledge injection into large language models (LLMs) via Low-Rank Adaptation (LoRA) fine-tuning. Method: Using Llama-3.1-8B-Instruct, we conduct systematic experiments across multi-scale rank/α configurations within a controlled factual injection framework, evaluating performance on MMLU, TruthfulQA, and answer distribution statistics. Contribution/Results: We identify—first time—a critical phenomenon: “known + novel fact” mixed training improves factual recall but degrades out-of-distribution question-answering performance, induces answer distribution shift, reduces refusal rates, and elevates confidence abnormally (impairing calibration). Knowledge injection exhibits a sharp capacity threshold; while mixed strategies maximize knowledge coverage, they trigger generalization degradation. Moreover, entity-level bias leads to answer overfitting. These findings expose intrinsic tensions in LoRA-based knowledge updating, providing theoretical grounding and practical guidelines for safe, predictable knowledge editing in LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The performance of Large Language Models (LLMs) on many tasks is greatly limited by the knowledge learned during pre-training and stored in the model's parameters. Low-rank adaptation (LoRA) is a popular and efficient training technique for updating or domain-specific adaptation of LLMs. In this study, we investigate how new facts can be incorporated into the LLM using LoRA without compromising the previously learned knowledge. We fine-tuned Llama-3.1-8B-instruct using LoRA with varying amounts of new knowledge. Our experiments have shown that the best results are obtained when the training data contains a mixture of known and new facts. However, this approach is still potentially harmful because the model's performance on external question-answering benchmarks declines after such fine-tuning. When the training data is biased towards certain entities, the model tends to regress to few overrepresented answers. In addition, we found that the model becomes more confident and refuses to provide an answer in only few cases. These findings highlight the potential pitfalls of LoRA-based LLM updates and underscore the importance of training data composition and tuning parameters to balance new knowledge integration and general model capabilities.

Problem

• Integrate new facts into LLMs using LoRA
• Balance new knowledge without harming pre-trained skills
• Address performance decline on external benchmarks

Innovation

• LoRA adapts LLMs efficiently
• Mix known and new facts
• Training data balance crucial