The Approximation Ratio for the Risk of Myopic Bayesian Active Learning for Linear Regression

2026-07-07

AI Summary

This work investigates the risk performance of greedy algorithms in myopic Bayesian active learning for linear regression. Viewing the greedy strategy as an approximation to optimal experimental design, the paper introduces—for the first time—the notion of a risk approximation ratio and establishes that this ratio is tightly bounded within an absolute constant factor and scales linearly with the maximum initial leverage score (MILS). Through a combination of theoretical analysis grounded in leverage score metrics and supporting numerical experiments, the study rigorously confirms the tightness of the risk bound for the greedy algorithm and reveals an intrinsic connection between its performance and the geometric structure of the data.

Abstract

Active learning studies the fundamental question: what data should we choose to observe? The greedy algorithm in optimal experiment design is a common heuristic and also equivalent to myopic Bayesian active learning for linear regression, the common framework where long-term planning is replaced with the one-step optimal choice. In this work, we prove a first-of-its-kind approximation ratio for the greedy algorithm's risk that is tight up to an absolute constant. The approximation ratio is linear in the maximum initial leverage score (MILS), a newly identified quantity fundamental to the greedy algorithm's performance. Finally, we illustrate the results with simple numerical simulations.

Problem

• active learning
• linear regression
• approximation ratio
• greedy algorithm
• Bayesian active learning

Innovation

• approximation ratio
• myopic Bayesian active learning
• greedy algorithm
• maximum initial leverage score
• linear regression