How Early Is Early Enough? Design-Dependent Observation-Window Sufficiency in Subscription Churn Prediction

2026-07-01

AI Summary

This study investigates the sufficiency of behavioral observation windows in subscription churn prediction, challenging the common assumption of fixed-time windows. Leveraging real-world music streaming data from KKBox, the authors conduct systematic evaluations through multi-window comparative experiments, stress tests, and precision-recall (PR) metrics across varying cohort definitions, churn criteria, and feature sets. Their findings reveal that the effectiveness of an observation window is highly contingent on task design: among high-churn users with manual renewals, 120 days of behavioral data yields a PR gain of 0.10, with diminishing returns emerging between 45–90 days—though this pattern shifts markedly under different experimental configurations. The paper underscores the necessity of explicitly specifying cohort composition, prediction targets, and feature definitions when selecting observation windows, offering methodological guidance for churn prediction research.

Abstract

How many days of early behavior suffice for subscription churn prediction? In the public KKBox dataset, the early indicator of churn is typically an indicator of someone's contract status; however, when looking in the heavily churned manual-renewal segment, having access to early behavior creates a substantial increase in prediction for that specific segment (PR +0.10 at 120 days). A nine-window sufficiency curve shows a diminishing-returns knee in a 45-90 day band. However, stress-testing over three cohort/task designs shows that this curve is singular to the design being tested; for example, in our test with a moving target, the curve inverts and can shift depending on the feature set used. Therefore, any window-sufficiency claim should state its cohort construction, target definition, and feature families. All evidence is from one music-streaming dataset; the mechanism should generalize but the magnitudes may not.

Problem

• churn prediction
• observation window
• early behavior
• cohort design
• subscription service

Innovation

• churn prediction
• observation window
• design dependency
• sufficiency curve
• cohort design