Reconciling Universal and Uniform Learning with $Q$-Aggregation

2026-09-04

AI Summary

研究通过Q-聚合方法解决了在有界响应回归中同时实现最小最大风险和普遍指数学习率的问题。

Abstract

We study regression under bounded responses in terms of excess mean squared error. When the comparator class is finite, this setting is known as model selection aggregation, and achieving minimax excess risk requires improper learning algorithms. Contrary to this, in the universal learning framework no improperness is needed, as simple empirical risk minimization achieves the best-possible exponential learning rate. Hence, the two frameworks suggest different optimal algorithmic principles. This poses the question of best-of-both-worlds guarantees: Are minimax and universal exponential rates achievable by the same algorithm? For finite hypothesis classes, we answer this question in the affirmative by showing that the $Q$-aggregation estimator - which is known to achieve minimax optimal tails - achieves exponential universal rates. A wide range of other estimators and algorithmic principles (ERM, sequential averaging, pruning, and star estimation) do not achieve both. For countably infinite hypothesis classes, we answer the question in the negative by showing that there is an inherent trade-off between achieving exponential universal and minimax uniform rates. This trade-off is exactly traced by combining optimal algorithms from each world using $Q$-aggregation. Besides these results, we prove several additional structural results about universal rates in learning with squared loss.

Problem

• regression
• excess mean squared error
• minimax excess risk
• universal learning
• exponential learning rate

Innovation

• Q-aggregation
• minimax optimal tails
• exponential universal rates
• finite hypothesis classes
• countably infinite hypothesis classes