A comparative analysis of machine learning algorithms for predicting probabilities of default

2025-06-24

AI Summary

This study addresses the insufficient predictive accuracy of probability of default (PD) estimation in credit risk assessment. It systematically evaluates six models—random forest, XGBoost, gradient boosting, AdaBoost, decision tree, and logistic regression—on the publicly available Scheule benchmark credit dataset. A unified evaluation framework is employed, incorporating AUC, Kolmogorov–Smirnov (KS) statistic, Brier score, and interpretability analysis. Results demonstrate that ensemble methods substantially outperform traditional logistic regression: XGBoost achieves the highest AUC (0.823) and exhibits robust generalization, while random forest strikes the optimal balance between feature interpretability and model stability. The study empirically validates the efficacy of machine learning for PD modeling and proposes a principled model selection criterion that jointly optimizes predictive performance and regulatory compliance. This contributes actionable insights and methodological guidance for financial institutions seeking to develop high-reliability, auditable intelligent credit risk management systems.

Technology Category

• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Predicting the probability of default (PD) of prospective loans is a critical objective for financial institutions. In recent years, machine learning (ML) algorithms have achieved remarkable success across a wide variety of prediction tasks; yet, they remain relatively underutilised in credit risk analysis. This paper highlights the opportunities that ML algorithms offer to this field by comparing the performance of five predictive models-Random Forests, Decision Trees, XGBoost, Gradient Boosting and AdaBoost-to the predominantly used logistic regression, over a benchmark dataset from Scheule et al. (Credit Risk Analytics: The R Companion). Our findings underscore the strengths and weaknesses of each method, providing valuable insights into the most effective ML algorithms for PD prediction in the context of loan portfolios.

Problem

• Compare ML algorithms for loan default prediction
• Evaluate performance against logistic regression
• Identify best ML methods for credit risk

Innovation

• Compares five ML algorithms for default prediction
• Benchmarks against traditional logistic regression
• Evaluates effectiveness using a credit risk dataset