When FinTech Meets Privacy: Securing Financial LLMs with Differential Private Fine-Tuning

2025-09-10

AI Summary

To address privacy risks posed by sensitive financial data in edge-deployed large language models (LLMs), this paper proposes a lightweight privacy-enhanced language model tailored for financial applications. Methodologically, it innovatively integrates differentially private fine-tuning with a streamlined network architecture to achieve both strong privacy guarantees and efficient on-device inference. The key contributions are: (1) the first systematic integration of rigorous differential privacy—formally satisfying $(varepsilon,delta)$-DP—into the LLM deployment pipeline for financial edge computing; (2) achieving performance comparable to full fine-tuning while maintaining model compactness (e.g., <50M parameters); and (3) empirical validation across multiple financial sentiment analysis benchmarks, demonstrating significantly stronger privacy protection than existing baselines and only marginal accuracy degradation (average drop <1.2%).

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The integration of Large Language Models (LLMs) into financial technology (FinTech) has revolutionized the analysis and processing of complex financial data, driving advancements in real-time decision-making and analytics. With the growing trend of deploying AI models on edge devices for financial applications, ensuring the privacy of sensitive financial data has become a significant challenge. To address this, we propose DPFinLLM, a privacy-enhanced, lightweight LLM specifically designed for on-device financial applications. DPFinLLM combines a robust differential privacy mechanism with a streamlined architecture inspired by state-of-the-art models, enabling secure and efficient processing of financial data. This proposed DPFinLLM can not only safeguard user data from privacy breaches but also ensure high performance across diverse financial tasks. Extensive experiments on multiple financial sentiment datasets validate the effectiveness of DPFinLLM, demonstrating its ability to achieve performance comparable to fully fine-tuned models, even under strict privacy constraints.

Problem

• Securing financial LLMs with differential privacy
• Protecting sensitive financial data on edge devices
• Ensuring high performance under privacy constraints

Innovation

• Differential privacy mechanism integration
• Lightweight architecture for on-device deployment
• Privacy-preserving financial data processing