FinTSB: A Comprehensive and Practical Benchmark for Financial Time Series Forecasting

2025-02-26

AI Summary

This paper addresses three critical challenges in financial time series forecasting (FinTSF): insufficient coverage of market movement patterns, inconsistent evaluation metrics, and misalignment with real-world trading scenarios. To this end, we propose FinTSB—the first comprehensive, practical benchmark for FinTSF. Methodologically, we introduce a novel four-category classification scheme for stock price movements; establish a standardized, three-dimensional evaluation protocol encompassing data, models, and metrics; and incorporate realistic market constraints—including transaction costs and price limits—to enable profit-oriented, backtested trading simulation. Technical contributions include: (i) a sequence-feature-driven data quality assessment framework; (ii) a multi-backbone, lightweight unified modeling pipeline; and (iii) a market-structure-aware profit-based evaluation framework. Experiments demonstrate that FinTSB significantly enhances cross-market-state model comparability and deployability, provides actionable model selection guidance, and is fully open-sourced—including code and data processing pipelines.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Game Theory and Economic Paradigms: Auctions and Market-Based Systems
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Financial time series (FinTS) record the behavior of human-brain-augmented decision-making, capturing valuable historical information that can be leveraged for profitable investment strategies. Not surprisingly, this area has attracted considerable attention from researchers, who have proposed a wide range of methods based on various backbones. However, the evaluation of the area often exhibits three systemic limitations: 1. Failure to account for the full spectrum of stock movement patterns observed in dynamic financial markets. (Diversity Gap), 2. The absence of unified assessment protocols undermines the validity of cross-study performance comparisons. (Standardization Deficit), and 3. Neglect of critical market structure factors, resulting in inflated performance metrics that lack practical applicability. (Real-World Mismatch). Addressing these limitations, we propose FinTSB, a comprehensive and practical benchmark for financial time series forecasting (FinTSF). To increase the variety, we categorize movement patterns into four specific parts, tokenize and pre-process the data, and assess the data quality based on some sequence characteristics. To eliminate biases due to different evaluation settings, we standardize the metrics across three dimensions and build a user-friendly, lightweight pipeline incorporating methods from various backbones. To accurately simulate real-world trading scenarios and facilitate practical implementation, we extensively model various regulatory constraints, including transaction fees, among others. Finally, we conduct extensive experiments on FinTSB, highlighting key insights to guide model selection under varying market conditions. Overall, FinTSB provides researchers with a novel and comprehensive platform for improving and evaluating FinTSF methods. The code is available at https://github.com/TongjiFinLab/FinTSBenchmark.

Problem

• Addressing diversity gap in stock movement patterns
• Standardizing evaluation protocols for cross-study comparisons
• Modeling real-world regulatory constraints for practical applicability

Innovation

• Standardized metrics for evaluation
• Tokenized and pre-processed data
• Modeled real-world trading constraints