e-Profits: A Business-Aligned Evaluation Metric for Profit-Sensitive Customer Churn Prediction

2025-07-09

AI Summary

Traditional churn prediction models rely on statistical metrics (e.g., AUC, F1) that ignore financial impact, leading to profit-suboptimal strategic decisions. To address this, we propose e-Profits—the first business-value-oriented evaluation metric—integrating customer lifetime value, retention probability, and intervention cost to enable personalized retention-rate estimation at both individual and segment levels. Methodologically, we combine Kaplan–Meier survival analysis with six classification algorithms and empirically validate our framework on the IBM Telco and Maven Telecom datasets. Results demonstrate that e-Profits substantially reshapes model ranking: it identifies models with moderate statistical performance but high profitability potential, uncovering previously hidden optimization opportunities for high-value customers overlooked by conventional metrics. We release an open-source toolkit supporting post-hoc profit attribution analysis.

Technology Category

• Machine Learning: Evaluation and Analysis
• Search and Optimization: Evaluation and Analysis
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Retention campaigns in customer relationship management often rely on churn prediction models evaluated using traditional metrics such as AUC and F1-score. However, these metrics fail to reflect financial outcomes and may mislead strategic decisions. We introduce e-Profits, a novel business-aligned evaluation metric that quantifies model performance based on customer-specific value, retention probability, and intervention costs. Unlike existing profit-based metrics such as Expected Maximum Profit, which assume fixed population-level parameters, e-Profits uses Kaplan-Meier survival analysis to estimate personalised retention rates and supports granular, per customer evaluation. We benchmark six classifiers across two telecom datasets (IBM Telco and Maven Telecom) and demonstrate that e-Profits reshapes model rankings compared to traditional metrics, revealing financial advantages in models previously overlooked by AUC or F1-score. The metric also enables segment-level insight into which models maximise return on investment for high-value customers. e-Profits is designed as an understandable, post hoc tool to support model evaluation in business contexts, particularly for marketing and analytics teams prioritising profit-driven decisions. All source code is available at: https://github.com/matifq/eprofits.

Problem

• Develops a profit-sensitive metric for churn prediction models
• Replaces traditional metrics like AUC with financial outcome focus
• Estimates personalized retention rates using survival analysis

Innovation

• Business-aligned metric e-Profits evaluates churn models
• Uses Kaplan-Meier for personalized retention rate estimation
• Enables granular ROI insights for high-value customers