PRAGMA: Revolut Foundation Model

2026-04-09

AI Summary

This work addresses the challenge of learning generalizable and effective financial representations from multi-source, heterogeneous, discrete, and variable-length bank event sequences—a setting where existing methods struggle. To this end, we propose PRAGMA, the first foundation model tailored for such data in finance. Built upon the Transformer architecture, PRAGMA employs a self-supervised masked modeling objective specifically designed for discrete, variable-length event sequences to enable unified representation learning across diverse downstream tasks. Through linear probing or lightweight fine-tuning, PRAGMA significantly outperforms current approaches on tasks including credit scoring, fraud detection, and customer lifetime value prediction. Notably, it achieves strong performance even with simple linear models, demonstrating the generality and transferability of its learned representations.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Discourse, Pragmatics & Argument Mining
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Modern financial systems generate vast quantities of transactional and event-level data that encode rich economic signals. This paper presents PRAGMA, a family of foundation models for multi-source banking event sequences. Our approach pre-trains a Transformer-based architecture with masked modelling on a large-scale, heterogeneous banking event corpus using a self-supervised objective tailored to the discrete, variable-length nature of financial records. The resulting model supports a wide range of downstream tasks such as credit scoring, fraud detection, and lifetime value prediction: strong performance can be achieved by training a simple linear model on top of the extracted embeddings and can be further improved with lightweight fine-tuning. Through extensive evaluation on downstream tasks, we demonstrate that PRAGMA achieves superior performance across multiple domains directly from raw event sequences, providing a general-purpose representation layer for financial applications.

Problem

• foundation model
• banking event sequences
• financial data representation
• multi-source financial data
• transactional data modeling

Innovation

• foundation model
• masked modeling
• financial event sequences
• self-supervised learning
• Transformer