Sample Efficient Omniprediction and Downstream Swap Regret for Non-Linear Losses

2025-02-18

AI Summary

This paper studies decision swap regret minimization for arbitrary multidimensional Lipschitz loss functions under both online adversarial and batch settings, unifying the problem as *joint optimization of downstream swap regret under generalized prediction*. We propose the first generic algorithm supporting nonlinear losses, multidimensional outcome spaces, and arbitrary downstream surrogates. We establish the first polynomial sample complexity bounds for arbitrary multidimensional Lipschitz losses—breaking prior exponential dependencies. For three canonical economic utility functions—Constant Elasticity of Substitution (CES), Cobb-Douglas, and Leontief—we derive tight regret and sample complexity bounds that are either dimension-free or exhibit only weak dependence on dimension. Our approach integrates techniques from online learning, online-to-batch conversion, multidimensional Lipschitz modeling, and structured utility analysis, achieving simultaneous advances in theoretical guarantees and practical applicability.

Technology Category

• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Adversarial Learning
• Machine Learning: Online Learning & Bandits

Application Category

• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We define"decision swap regret"which generalizes both prediction for downstream swap regret and omniprediction, and give algorithms for obtaining it for arbitrary multi-dimensional Lipschitz loss functions in online adversarial settings. We also give sample complexity bounds in the batch setting via an online-to-batch reduction. When applied to omniprediction, our algorithm gives the first polynomial sample-complexity bounds for Lipschitz loss functions -- prior bounds either applied only to linear loss (or binary outcomes) or scaled exponentially with the error parameter even under the assumption that the loss functions were convex. When applied to prediction for downstream regret, we give the first algorithm capable of guaranteeing swap regret bounds for all downstream agents with non-linear loss functions over a multi-dimensional outcome space: prior work applied only to linear loss functions, modeling risk neutral agents. Our general bounds scale exponentially with the dimension of the outcome space, but we give improved regret and sample complexity bounds for specific families of multidimensional functions of economic interest: constant elasticity of substitution (CES), Cobb-Douglas, and Leontief utility functions.

Problem

• Generalize decision swap regret for non-linear loss functions
• Provide polynomial sample-complexity for Lipschitz loss functions
• Guarantee swap regret bounds for multi-dimensional outcome spaces

Innovation

• Defines decision swap regret generalization
• Polynomial sample-complexity for Lipschitz functions
• Guarantees swap regret for non-linear losses