A Foundation Model for Multimodal Event Sequences in Financial Applications

2026-07-10

AI Summary

This work addresses the limitations of traditional financial forecasting approaches, which rely on task-specific models and handcrafted features, struggle to integrate heterogeneous data (e.g., transactional and digital interaction logs), and exhibit poor reusability. The authors propose the first multimodal event-sequence foundation model tailored for financial applications. It unifies users’ multisource behaviors into temporal sequences and leverages a Transformer architecture pretrained via next-event prediction to learn general-purpose representations. These representations, combined with existing features, enable lightweight fine-tuning across diverse downstream tasks. The model achieves early fusion of multimodal events and cross-task representation sharing, effectively breaking down task silos. Deployed at a major bank in Eastern Europe, it significantly outperforms conventional models, enhancing key business metrics while reducing development costs.

Abstract

Predictive modeling is a core component of modern financial services, where a wide range of tasks are traditionally addressed using separate models trained on manually engineered tabular features. This task-specific approach limits reuse and makes it difficult to fully exploit heterogeneous data sources such as transaction histories and digital interaction signals. In this paper, we present an approach based on pretraining a foundation transformer model on multimodal sequences of user events. Events from multiple data sources are unified into a single chronological sequence, enabling early fusion of heterogeneous modalities and learning of general-purpose representations via a next-event prediction objective. These representations are combined with existing engineered user features, on top of which lightweight neural models are trained for multiple downstream tasks. The proposed system outperforms traditional task-specific models while reducing development overhead. The approach was deployed in production at one of the biggest banks in Eastern Europe, resulting in measurable improvements in business metrics.

Problem

• predictive modeling
• financial applications
• heterogeneous data
• task-specific models
• multimodal event sequences

Innovation

• foundation model
• multimodal event sequences
• early fusion
• next-event prediction
• financial applications