Personalizing black-box models for nonparametric regression with minimax optimality

2026-01-04

AI Summary

This work addresses the challenge of effectively leveraging pre-trained black-box models for personalized nonparametric regression in data-scarce settings. The authors develop a theoretical framework and propose a nonparametric regression algorithm that integrates information from black-box models while maintaining robustness even when these pre-trained models fail. Under limited sample regimes, the proposed method achieves the minimax-optimal convergence rate established within the framework. The theoretical analysis draws upon nonparametric regression theory, minimax theory, and statistical learning theory. Empirical evaluations on both synthetic data and the California housing dataset demonstrate the algorithm’s superior performance in low-sample scenarios, confirming its practical efficacy and theoretical guarantees.

Technology Category

• Machine Learning: Learning Theory
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advances in large-scale models, including deep neural networks and large language models, have substantially improved performance across a wide range of learning tasks. The widespread availability of such pre-trained models creates new opportunities for data-efficient statistical learning, provided they can be effectively integrated into downstream tasks. Motivated by this setting, we study few-shot personalization, where a pre-trained black-box model is adapted to a target domain using a limited number of samples. We develop a theoretical framework for few-shot personalization in nonparametric regression and propose algorithms that can incorporate a black-box pre-trained model into the regression procedure. We establish the minimax optimal rate for the personalization problem and show that the proposed method attains this rate. Our results clarify the statistical benefits of leveraging pre-trained models under sample scarcity and provide robustness guarantees when the pre-trained model is not informative. We illustrate the finite-sample performance of the methods through simulations and an application to the California housing dataset with several pre-trained models.

Problem

• few-shot personalization
• black-box models
• nonparametric regression
• pre-trained models
• minimax optimality

Innovation

• few-shot personalization
• minimax optimality
• nonparametric regression
• black-box models
• pre-trained models