Which Directions Matter? Sparse Design for Affine Robust Optimization

2026-06-12

AI Summary

This work addresses the challenge of efficiently selecting key uncertainty directions from a finite dictionary to construct computationally tractable robust optimization uncertainty sets. It proposes a data-driven approach that builds atomic uncertainty sets with closed-form support functions by identifying a sparse subset of atoms capable of covering critical evaluation directions—such as gradients, adversarial perturbations, or distributional shifts. The core contribution lies in the design of a monotone and submodular coverage objective, which enables a greedy algorithm with a provable $(1 - 1/e)$ approximation guarantee. The method also provides out-of-sample performance loss bounds and a radius calibration rule, achieving both theoretical approximation guarantees and significant improvements in scalability and out-of-sample robustness.

Abstract

Robust machine learning and optimization rely on the uncertainty model choice. We investigate which uncertainty directions a model must cover when defined by a finite dictionary and a budget constraint. Selecting a subset forms an atomic uncertainty set with a closed form support function, yielding tractable robust programs for affine objectives. We propose a data driven selection rule based on a coverage objective over evaluation directions, including gradients, adversarial perturbations, or shifts observed on held out data. We prove this objective is monotone and submodular, supporting a greedy method with a $(1-1/e)$ approximation guarantee and a matching hardness barrier. We also provide a certificate bounding the loss from the selected subset and a radius calibration rule with out of sample control.

Problem

• Robust Optimization
• Uncertainty Directions
• Sparse Design
• Affine Objectives
• Submodular Optimization

Innovation

• sparse design
• affine robust optimization
• submodular optimization
• uncertainty set
• data-driven selection