Tight Bounds for Data-driven Multiple Hyper-parameter Tuning with Structured Loss Function

2026-08-18

AI Summary

本文通过实代数几何和多机制下界框架，解决了数据驱动的多超参数调优中紧致边界问题，提高了样本复杂度估计精度。

Abstract

Data-driven algorithm design frames hyperparameter tuning as a statistical learning problem, but establishing generalization guarantees remains challenging due to the implicit, non-smooth dependence of model performance on hyperparameters. Existing multi-dimensional bounds under piecewise-polynomial assumptions remain theoretically loose and lack comprehensive lower bounds. We resolve this by establishing tight pseudo-dimension bounds for multi-dimensional data-driven tuning. First, we refine the learning-theoretic upper bound using real algebraic geometry; by analyzing invariant connected sign cells during block elimination rather than isolated sign vectors, we avoid topological over-counting to derive strictly sharper sample complexities. Second, we present a multi-regime lower-bound framework that disentangles combinatorial and algebraic capacities. By constructing shattered problem instances across distinct regimes, we prove our upper bounds are tightly saturated. Finally, we extend our topological framework to accommodate general bi-level validation-loss tuning and broader semi-algebraic applications.

Problem

• hyperparameter tuning
• generalization guarantees
• piecewise-polynomial assumptions
• pseudo-dimension bounds
• bi-level validation-loss tuning

Innovation

• pseudo-dimension bounds
• real algebraic geometry
• multi-regime lower-bound framework
• bi-level validation-loss tuning
• semi-algebraic applications