Stock Market Prediction Using Node Transformer Architecture Integrated with BERT Sentiment Analysis

2026-03-06

AI Summary

This study addresses the challenges of high noise, non-stationarity, and complex inter-stock dependencies in financial markets by proposing a novel stock price prediction framework that integrates graph structures with multi-source information. The method constructs a stock relationship graph and employs a node-wise Transformer to jointly model temporal and cross-sectional dynamic dependencies. It further incorporates sentiment features extracted from social media via a fine-tuned BERT model, fused through an attention mechanism for multimodal integration. To the best of our knowledge, this is the first work to combine node Transformers with financial sentiment analysis, explicitly capturing multidimensional inter-stock relationships. Evaluated on S&P 500 data, the approach achieves a MAPE of 0.80%, significantly outperforming ARIMA and LSTM baselines. The inclusion of sentiment features reduces overall prediction error by 10% (up to 25% during earnings seasons), yields a directional accuracy of 65%, and maintains robustness with MAPE consistently below 1.5% even during high-volatility periods.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Stock market prediction presents considerable challenges for investors, financial institutions, and policymakers operating in complex market environments characterized by noise, non-stationarity, and behavioral dynamics. Traditional forecasting methods often fail to capture the intricate patterns and cross-sectional dependencies inherent in financial markets. This paper presents an integrated framework combining a node transformer architecture with BERT-based sentiment analysis for stock price forecasting. The proposed model represents the stock market as a graph structure where individual stocks form nodes and edges capture relationships including sectoral affiliations, correlated price movements, and supply chain connections. A fine-tuned BERT model extracts sentiment from social media posts and combines it with quantitative market features through attention-based fusion. The node transformer processes historical market data while capturing both temporal evolution and cross-sectional dependencies among stocks. Experiments on 20 S&P 500 stocks spanning January 1982 to March 2025 demonstrate that the integrated model achieves a mean absolute percentage error (MAPE) of 0.80% for one-day-ahead predictions, compared to 1.20% for ARIMA and 1.00% for LSTM. Sentiment analysis reduces prediction error by 10% overall and 25% during earnings announcements, while graph-based modeling contributes an additional 15% improvement by capturing inter-stock dependencies. Directional accuracy reaches 65% for one-day forecasts. Statistical validation through paired t-tests confirms these improvements (p<0.05 for all comparisons). The model maintains MAPE below 1.5% during high-volatility periods where baseline models exceed 2%.

Problem

• Stock Market Prediction
• Non-stationarity
• Cross-sectional Dependencies
• Behavioral Dynamics
• Financial Forecasting

Innovation

• Node Transformer
• BERT Sentiment Analysis
• Graph-based Stock Modeling
• Attention-based Fusion
• Cross-sectional Dependency