Capacity Constraints Make Admissions Processes Less Predictable

2026-01-16

AI Summary

This study addresses the challenge posed by capacity constraints in admissions processes, which render admission outcomes sensitive to the composition of applicant pools and thereby severely degrade the generalization of conventional machine learning models under distributional shift. For the first time, the paper formally defines two theoretical concepts—“instability” and “variability”—within the context of admissions mechanisms, elucidating how capacity limits fundamentally undermine predictive transferability. Leveraging individual-level admissions data from New York City high schools, the authors combine theoretical analysis with empirical evaluation to systematically assess how model performance deteriorates across varying applicant populations under different decision rules. The findings demonstrate that greater divergence between training and test populations leads to more pronounced performance degradation, with schools exhibiting high instability or variability suffering the most severe declines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Machine learning models are often used to make predictions about admissions process outcomes, such as for colleges or jobs. However, such decision processes differ substantially from the conventional machine learning paradigm. Because admissions decisions are capacity-constrained, whether a student is admitted depends on the other applicants who apply. We show how this dependence affects predictive performance even in otherwise ideal settings. Theoretically, we introduce two concepts that characterize the relationship between admission function properties, machine learning representation, and generalization to applicant pool distribution shifts: instability, which measures how many existing decisions can change when a single new applicant is introduced; and variability, which measures the number of unique students whose decisions can change. Empirically, we illustrate our theory on individual-level admissions data from the New York City high school matching system, showing that machine learning performance degrades as the applicant pool increasingly differs from the training data. Furthermore, there are larger performance drops for schools using decision rules that are more unstable and variable. Our work raises questions about the reliability of predicting individual admissions probabilities.

Problem

• capacity constraints
• admissions prediction
• distribution shift
• machine learning reliability
• applicant pool

Innovation

• capacity constraints
• instability
• variability
• admissions prediction
• distribution shift