Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation

2024-11-07

arXiv.org

AI Summary

To address the performance instability of Low-Rank Adaptation (LoRA) in large language model fine-tuning—stemming from its sensitivity to hyperparameters—this paper proposes MonteCLoRA, a low-rank adaptation method grounded in Bayesian reparameterization and Monte Carlo estimation. Its key innovation is the first integration of hyperpriors into LoRA, enabling robust learning of low-rank parameters via unbiased, low-variance posterior sampling while introducing only O(1) additional parameters. On natural language understanding tasks using RoBERTa-base, MonteCLoRA achieves a 3.8% absolute accuracy gain and an 8.6% improvement in robustness. In zero-shot generation with LLaMA-1-7B, it reduces output variance by 50%. Overall, MonteCLoRA significantly enhances both the stability and generalization capability of parameter-efficient fine-tuning.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) are highly resource-intensive to fine-tune due to their enormous size. While low-rank adaptation is a prominent parameter-efficient fine-tuning approach, it suffers from sensitivity to hyperparameter choices, leading to instability in model performance on fine-tuning downstream tasks. This paper highlights the importance of effective parameterization in low-rank fine-tuning to reduce estimator variance and enhance the stability of final model outputs. We propose MonteCLoRA, an efficient fine-tuning technique, employing Monte Carlo estimation to learn an unbiased posterior estimation of low-rank parameters with low expected variance, which stabilizes fine-tuned LLMs with only O(1) additional parameters. MonteCLoRA shows significant improvements in accuracy and robustness, achieving up to 3.8% higher accuracy and 8.6% greater robustness than existing efficient fine-tuning methods on natural language understanding tasks with pre-trained RoBERTa-base. Furthermore, in generative tasks with pre-trained LLaMA-1-7B, MonteCLoRA demonstrates robust zero-shot performance with 50% lower variance than the contemporary efficient fine-tuning methods. The theoretical and empirical results presented in the paper underscore how parameterization and hyperpriors balance exploration-exploitation in the low-rank parametric space, therefore leading to more optimal and robust parameter estimation during efficient fine-tuning.

Problem

• Reducing sensitivity to hyperparameters in low-rank adaptation
• Enhancing stability of fine-tuned LLM outputs
• Improving accuracy and robustness in efficient fine-tuning

Innovation

• Bayesian reparameterization for low-rank adaptation
• Monte Carlo estimation for unbiased posterior
• O(r) parameters for stable fine-tuning