Understanding Overparametrization in Survival Models through Double-Descent

2025-12-13

AI Summary

The double-descent phenomenon in overparameterized models remains unexplored in survival analysis, despite its documented presence in classification and regression. Method: We conduct the first systematic investigation of double descent in survival modeling, focusing on four representative architectures—DeepSurv, PC-Hazard, Nnet-Survival, and N-MTLR. We rigorously define interpolation and finite-norm interpolation conditions, analyzing their feasibility under likelihood loss structure and model implementation constraints. Our approach combines theoretical analysis (existence proofs for interpolation, generalization error decomposition), capacity-sweep experiments across multiple survival datasets, and validation under both Cox proportional hazards and discrete-time hazard frameworks. Results: (1) Double descent is not universally observed in survival models; (2) finite-norm interpolation is well-defined and empirically verifiable; (3) overparameterization typically exacerbates—not reduces—the generalization gap, refuting the existence of “benign overfitting” in this domain. These findings challenge the common assumption that double-descent behavior from standard supervised learning tasks directly transfers to survival analysis.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Search and Optimization: Non-convex Optimization

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Classical statistical learning theory predicts a U-shaped relationship between test loss and model capacity, driven by the bias-variance trade-off. Recent advances in modern machine learning have revealed a more complex pattern, double-descent, in which test loss, after peaking near the interpolation threshold, decreases again as model capacity continues to grow. While this behavior has been extensively analyzed in regression and classification, its manifestation in survival analysis remains unexplored. This study investigates double-descent in four representative survival models: DeepSurv, PC-Hazard, Nnet-Survival, and N-MTLR. We rigorously define interpolation and finite-norm interpolation, two key characteristics of loss-based models to understand double-descent. We then show the existence (or absence) of (finite-norm) interpolation of all four models. Our findings clarify how likelihood-based losses and model implementation jointly determine the feasibility of interpolation and show that overfitting should not be regarded as benign for survival models. All theoretical results are supported by numerical experiments that highlight the distinct generalization behaviors of survival models.

Problem

• Investigates double-descent phenomenon in survival analysis models
• Defines interpolation and finite-norm interpolation for loss-based models
• Examines how likelihood losses and implementation affect interpolation feasibility

Innovation

• Defined interpolation and finite-norm interpolation for survival models
• Analyzed double-descent in four survival models including DeepSurv
• Showed overfitting is not benign in survival analysis models