Distributionally Robust Linear Regression With Block Lewis Weights

2026-06-30

AI Summary

This work addresses the grouped distributionally robust (GDR) least squares problem, which seeks to minimize the worst-case loss across multiple data groups. The authors introduce block Lewis weights—a novel geometric tool—to reformulate the problem as a specially weighted least squares instance. By integrating an accelerated proximal algorithm with a structured linear system solver tailored for systems of the form \(A^\top B A\), they achieve an efficient solution method that unifies optimization frameworks for both average and robust losses. The proposed approach outperforms interior-point methods at moderate accuracy levels. Theoretically, it attains a \((1+\varepsilon)\)-approximate solution using only \(\widetilde{O}(\min\{\mathrm{rank}(A), m\}^{1/3} \varepsilon^{-2/3})\) linear system solves, yielding the current best-known guarantee for the special case of \(\ell_\infty\) regression.

Abstract

We present an algorithm for the group distributionally robust (GDR) least squares problem. Given $m$ groups, a parameter vector in $\mathbb{R}^d$, and stacked design matrices and responses $\mathbf{A}$ and $\mathbf{b}$, our algorithm obtains a $(1+\varepsilon)$-multiplicative optimal solution using $\widetilde{O}(\min\{\mathsf{rank}(\mathbf{A}),m\}^{1/3}\varepsilon^{-2/3})$ linear-system-solves of matrices of the form $\mathbf{A}^{\top}\mathbf{B}\mathbf{A}$ for block-diagonal $\mathbf{B}$. Our technical methods follow from a recent geometric construction, block Lewis weights, that relates the empirical GDR problem to a carefully chosen least squares problem and an application of accelerated proximal methods. Our algorithm improves over known interior point methods for moderate accuracy regimes and matches the state-of-the-art guarantees for the special case of $\ell_{\infty}$ regression. We also give algorithms that smoothly interpolate between minimizing the average least squares loss and the distributionally robust loss.

Problem

• distributionally robust optimization
• linear regression
• group robustness
• least squares
• block Lewis weights

Innovation

• distributionally robust optimization
• block Lewis weights
• accelerated proximal methods
• group robust regression
• linear system solves