A Set-Sequence Model for Time Series

2025-05-16

AI Summary

Financial time-series forecasting faces challenges in jointly modeling individual units (e.g., stocks, loans) influenced by unit-specific micro-features, macroeconomic variables, and latent cross-sectional dependencies. To address this, we propose a joint-learning Set-Sequence architecture that represents the cross-sectional set as a learnable shared summary. A set-invariant encoder generates a linear-complexity, variable-length summary automatically, seamlessly integrating with any sequential model (e.g., LSTM or Transformer) without manual feature engineering. Our key contribution is the first end-to-end integration of cross-sectional dependency modeling into the training pipeline, enabling concurrent learning of micro-level, macro-level, and inter-unit effects. Extensive experiments on stock return prediction and mortgage borrower behavior modeling demonstrate significant improvements over state-of-the-art baselines, validating the model’s capacity to effectively capture implicit cross-sectional dependencies and its strong generalization performance.

Technology Category

• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In many financial prediction problems, the behavior of individual units (such as loans, bonds, or stocks) is influenced by observable unit-level factors and macroeconomic variables, as well as by latent cross-sectional effects. Traditional approaches attempt to capture these latent effects via handcrafted summary features. We propose a Set-Sequence model that eliminates the need for handcrafted features. The Set model first learns a shared cross-sectional summary at each period. The Sequence model then ingests the summary-augmented time series for each unit independently to predict its outcome. Both components are learned jointly over arbitrary sets sampled during training. Our approach harnesses the set nature of the cross-section and is computationally efficient, generating set summaries in linear time relative to the number of units. It is also flexible, allowing the use of existing sequence models and accommodating a variable number of units at inference. Empirical evaluations demonstrate that our Set-Sequence model significantly outperforms benchmarks on stock return prediction and mortgage behavior tasks. Code will be released.

Problem

• Modeling latent cross-sectional effects in financial prediction without handcrafted features
• Learning shared cross-sectional summaries and unit-specific time series jointly
• Improving prediction accuracy for stock returns and mortgage behavior tasks

Innovation

• Set-Sequence model eliminates handcrafted features
• Learns shared cross-sectional summaries efficiently
• Flexibly integrates existing sequence models