FinLoRA: Benchmarking LoRA Methods for Fine-Tuning LLMs on Financial Datasets

2025-05-26

AI Summary

Low-rank adaptation (LoRA) lacks systematic evaluation in financial-domain large language models (LLMs), particularly for high-stakes tasks such as CFA exam question answering and SEC filing analysis. Method: We introduce the first financial-specific LoRA benchmark, comprising 19 datasets—including four novel XBRL semantic parsing benchmarks—and conduct comprehensive evaluation across five mainstream LLMs and five LoRA variants. Our framework integrates XBRL-aware structured semantic modeling, multi-granularity task design, and joint BERTScore/F1 evaluation, while quantifying GPU memory footprint and training overhead. Contribution/Results: LoRA achieves an average 36% performance gain over baselines. All datasets, fine-tuned adapters, code, and documentation are publicly released, substantially lowering the barrier to financial LLM customization and advancing trustworthy, accessible AI deployment in finance.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Low-rank adaptation (LoRA) methods show great potential for scaling pre-trained general-purpose Large Language Models (LLMs) to hundreds or thousands of use scenarios. However, their efficacy in high-stakes domains like finance is rarely explored, e.g., passing CFA exams and analyzing SEC filings. In this paper, we present the open-source FinLoRA project that benchmarks LoRA methods on both general and highly professional financial tasks. First, we curated 19 datasets covering diverse financial applications; in particular, we created four novel XBRL analysis datasets based on 150 SEC filings. Second, we evaluated five LoRA methods and five base LLMs. Finally, we provide extensive experimental results in terms of accuracy, F1, and BERTScore and report computational cost in terms of time and GPU memory during fine-tuning and inference stages. We find that LoRA methods achieved substantial performance gains of 36% on average over base models. Our FinLoRA project provides an affordable and scalable approach to democratize financial intelligence to the general public. Datasets, LoRA adapters, code, and documentation are available at https://github.com/Open-Finance-Lab/FinLoRA

Problem

• Benchmarking LoRA methods for financial tasks
• Evaluating LoRA efficacy in high-stakes finance domains
• Providing affordable financial intelligence via LoRA

Innovation

• Benchmarking LoRA methods on financial datasets
• Evaluating five LoRA methods and five base LLMs
• Providing extensive experimental results and computational costs