Are all models wrong? Fundamental limits in distribution-free empirical model falsification

2025-02-10

AI Summary

This paper investigates the constructive feasibility of strictly positive, distribution-free lower bounds on model class risk—i.e., whether “all models must fail” is unavoidable. Focusing on interpolation regimes and related settings, it establishes the first model-agnostic, distribution-free impossibility theorem for constructing such risk lower bounds. Leveraging tools from statistical decision theory and minimax analysis, the authors prove that, given finite samples, no algorithm can reliably produce a strictly positive lower bound on risk for broad model classes—including linear regression and tree-based models. This result exposes a fundamental theoretical limitation of empirical falsification: empirically estimated lower bounds on the optimal test error of a model class are inherently unreliable—they cannot certify the existence of systematic bias within the class. As the first hardness theorem characterizing the non-constructibility of model-class risk lower bounds, this work provides a rigorous foundation for understanding the limits of model evaluation in distribution-free learning.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In statistics and machine learning, when we train a fitted model on available data, we typically want to ensure that we are searching within a model class that contains at least one accurate model -- that is, we would like to ensure an upper bound on the model class risk (the lowest possible risk that can be attained by any model in the class). However, it is also of interest to establish lower bounds on the model class risk, for instance so that we can determine whether our fitted model is at least approximately optimal within the class, or, so that we can decide whether the model class is unsuitable for the particular task at hand. Particularly in the setting of interpolation learning where machine learning models are trained to reach zero error on the training data, we might ask if, at the very least, a positive lower bound on the model class risk is possible -- or are we unable to detect that"all models are wrong"? In this work, we answer these questions in a distribution-free setting by establishing a model-agnostic, fundamental hardness result for the problem of constructing a lower bound on the best test error achievable over a model class, and examine its implications on specific model classes such as tree-based methods and linear regression.

Problem

• Establishing lower bounds on model class risk
• Determining model class suitability for tasks
• Analyzing fundamental limits in model falsification

Innovation

• Distribution-free empirical model falsification
• Model-agnostic hardness result
• Lower bounds on model class risk