Language Modeling for the Future of Finance: A Quantitative Survey into Metrics, Tasks, and Data Opportunities

2025-04-09

AI Summary

Financial natural language processing (FinNLP) lacks a systematic, large-scale empirical analysis of methodological trends, evaluation practices, and persistent challenges across diverse tasks and temporal contexts. Method: We conduct a comprehensive meta-analysis of 374 peer-reviewed FinNLP papers published between 2017–2024, covering 221 distinct tasks, and construct an 11-dimensional analytical framework encompassing model evolution, task distribution, data curation, and evaluation paradigms. Contribution/Results: This is the first quantitative study to characterize the transfer dynamics of general-purpose large language models (LLMs) into financial domains. We identify critical gaps in data accessibility, domain adaptation, and crisis-period robustness—and advocate for crisis-aware datasets and domain-specific evaluation metrics to enhance practical reliability. Key findings include sustained advances in sentiment analysis and information extraction, alongside rising emphasis on interpretability and privacy preservation. Our work provides empirically grounded guidance for redefining FinNLP evaluation standards, advancing data infrastructure, and shaping future methodology.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Evaluation and Analysis
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent advances in language modeling have led to growing interest in applying Natural Language Processing (NLP) techniques to financial problems, enabling new approaches to analysis and decision-making. To systematically examine this trend, we review 374 NLP research papers published between 2017 and 2024 across 38 conferences and workshops, with a focused analysis of 221 papers that directly address finance-related tasks. We evaluate these papers across 11 qualitative and quantitative dimensions, identifying key trends such as the increasing use of general-purpose language models, steady progress in sentiment analysis and information extraction, and emerging efforts around explainability and privacy-preserving methods. We also discuss the use of evaluation metrics, highlighting the importance of domain-specific ones to complement standard machine learning metrics. Our findings emphasize the need for more accessible, adaptive datasets and highlight the significance of incorporating financial crisis periods to strengthen model robustness under real-world conditions. This survey provides a structured overview of NLP research applied to finance and offers practical insights for researchers and practitioners working at this intersection.

Problem

• Surveying NLP applications in finance from 2017-2024
• Evaluating trends in financial NLP tasks and metrics
• Identifying needs for better datasets and model robustness

Innovation

• General-purpose language models for finance
• Domain-specific evaluation metrics importance
• Financial crisis data for robustness