Comparative analysis of financial data differentiation techniques using LSTM neural network

2025-05-25

AI Summary

Traditional log-return preprocessing for financial time series neglects long-memory properties, limiting model generalizability and trading strategy robustness. Method: This study systematically compares log-returns, fractional differencing (FD), and its temperature-truncated extension (T-FD) within LSTM-based forecasting and trading frameworks. FD orders are estimated via maximum likelihood estimation (MLE) or cross-validation (CV); technical indicators are integrated; and performance is evaluated using rolling-window backtesting with risk-adjusted metrics—including the Sharpe ratio. Contribution/Results: For the first time across multiple equity indices, FD and T-FD are empirically shown to significantly reduce LSTM forecasting error (average MAE reduction of 12.7%) and consistently improve Sharpe ratios—by a mean increase of 0.31—in both single-asset and index-portfolio trading strategies. These findings validate memory-preserving transformations as critical for enhancing model generalization and strategy robustness in financial time-series modeling.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We compare traditional approach of computing logarithmic returns with the fractional differencing method and its tempered extension as methods of data preparation before their usage in advanced machine learning models. Differencing parameters are estimated using multiple techniques. The empirical investigation is conducted on data from four major stock indices covering the most recent 10-year period. The set of explanatory variables is additionally extended with technical indicators. The effectiveness of the differencing methods is evaluated using both forecast error metrics and risk-adjusted return trading performance metrics. The findings suggest that fractional differentiation methods provide a suitable data transformation technique, improving the predictive model forecasting performance. Furthermore, the generated predictions appeared to be effective in constructing profitable trading strategies for both individual assets and a portfolio of stock indices. These results underline the importance of appropriate data transformation techniques in financial time series forecasting, supporting the application of memory-preserving techniques.

Problem

• Comparing financial data differentiation techniques for LSTM models
• Evaluating differencing methods using forecast and trading metrics
• Assessing impact of data transformation on predictive performance

Innovation

• Using LSTM neural network for financial data analysis
• Comparing logarithmic returns with fractional differencing methods
• Extending variables with technical indicators for better predictions