A Generative Approach to Credit Prediction with Learnable Prompts for Multi-scale Temporal Representation Learning

2024-04-19

AI Summary

To address the limitations of traditional statistical models—namely, heavy reliance on manual hyperparameter tuning—and the poor generalization of deep learning methods in long-horizon credit forecasting, this paper proposes the first generative framework tailored for industrial credit scoring. The method reframes credit scoring as a task of generating multi-scale future behavioral distributions for users, enabled by a Tabular-to-Sequence transformation that constructs temporal representations from static tabular data. It introduces a dual-granularity learnable prompting mechanism—operating at both feature and user levels—to jointly model cross-temporal dependencies. Furthermore, it proposes FinLangNet, a unified architecture that jointly encodes structured financial features and sequential semantic patterns. Evaluated in real-world production environments, the framework achieves a 7.2-point KS improvement and a 9.9% relative reduction in default rate. Extensive validation on the UEA benchmark confirms its strong generalizability and scalability across diverse time-series domains.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Feature Construction/Reformulation
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Recent industrial credit scoring models remain heavily reliant on manually tuned statistical learning methods. While deep learning offers promising solutions, its effectiveness is often limited by the complexity of financial data, particularly in long-horizon scenarios. In this work, we propose FinLangNet, which addresses credit scoring by reframing it as the task of generating multi-scale distributions of a user's future behavior. Within this framework, tabular data is transformed into sequential representations, enabling the generation of user embeddings across multiple temporal scales. Inspired by the recent success of prompt-based training in Large Language Models (LLMs), FinLangNet also introduces two types of prompts to model and capture user behavior at both the feature-granularity and user-granularity levels. Experimental results demonstrate that FinLangNet outperforms the online XGBoost benchmark, achieving a 7.2% improvement in KS metric performance and a 9.9% reduction in the relative bad debt rate. Furthermore, FinLangNet exhibits superior performance on public UEA archives, underscoring its scalability and adaptability in time series classification tasks.

Problem

• Improve credit scoring with multi-scale temporal learning
• Transform tabular data into sequential user embeddings
• Introduce prompt-based training for user behavior modeling

Innovation

• Generative credit prediction model
• Multi-scale temporal representation learning
• Prompt-based training for user behavior