Regression Model Selection Under General Conditions

2025-10-10

The Federal Reserve Bank of Kansas City Research Working Papers

AI Summary

Existing asymptotic optimality proofs for model selection criteria rely heavily on restrictive assumptions—specific model classes, estimation methods, and data structures—limiting their theoretical validity and practical applicability. Method: We develop a unified asymptotic theory framework that relaxes these constraints, accommodating diverse models (e.g., linear regression, quantile regression, penalized regression), heterogeneous data structures (independent, dependent, and high-dimensional), and broad estimation paradigms (maximum likelihood, generalized method of moments, linear smoothers). Contribution/Results: Within this general setting, we establish, for the first time, rigorous asymptotic optimality of canonical criteria—including AIC, BIC, and cross-validation—under minimal regularity conditions. Our results substantially broaden the theoretical scope of these criteria, enabling principled model selection in complex, real-world scenarios with dependent or high-dimensional data. The framework provides a robust, general foundation for statistical inference and model choice beyond conventional parametric and i.i.d. settings.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: User model development and evaluation

Abstract

Model selection criteria are one of the most important tools in statistics. Proofs showing a model selection criterion is asymptotically optimal are tailored to the type of model (linear regression, quantile regression, penalized regression, etc.), the estimation method (linear smoothers, maximum likelihood, generalized method of moments, etc.), the type of data (i.i.d., dependent, high dimensional, etc.), and the type of model selection criterion. Moreover, assumptions are often restrictive and unrealistic making it a slow and winding process for researchers to determine if a model selection criterion is selecting an optimal model. This paper provides general proofs showing asymptotic optimality for a wide range of model selection criteria under general conditions. This paper not only asymptotically justifies model selection criteria for most situations, but it also unifies and extends a range of previously disparate results.

Problem

• Proving asymptotic optimality for diverse model selection criteria
• Unifying disparate results across different statistical models
• Extending validity of criteria under general realistic conditions

Innovation

• General proofs for asymptotic optimality under broad conditions
• Unifies disparate model selection criteria results
• Extends applicability across various regression types