Defensive Generation

2026-02-24

AI Summary

This work addresses the problem of efficiently generating scalar, multiclass, and vector-valued outputs online that are unfalsifiable with respect to any arbitrary infinite test class—including those involving higher-order moments. By establishing a theoretical connection between online high-dimensional multicalibration and variational inequalities in expectation, the authors propose a near-linear-time algorithm grounded in reproducing kernel Hilbert spaces (RKHS). This method achieves, for the first time, online indistinguishability guarantees for non-Bernoulli outputs. The algorithm operates in near-linear time with respect to the number of samples, and its generation error converges at the optimal rate of \(T^{-1/2}\), substantially extending both the applicability and computational efficiency of unfalsifiable generative models.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Generation
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

We study the problem of efficiently producing, in an online fashion, generative models of scalar, multiclass, and vector-valued outcomes that cannot be falsified on the basis of the observed data and a pre-specified collection of computational tests. Our contributions are twofold. First, we expand on connections between online high-dimensional multicalibration with respect to an RKHS and recent advances in expected variational inequality problems, enabling efficient algorithms for the former. We then apply this algorithmic machinery to the problem of outcome indistinguishability. Our procedure, Defensive Generation, is the first to efficiently produce online outcome indistinguishable generative models of non-Bernoulli outcomes that are unfalsifiable with respect to infinite classes of tests, including those that examine higher-order moments of the generated distributions. Furthermore, our method runs in near-linear time in the number of samples and achieves the optimal, vanishing T^{-1/2} rate for generation error.

Problem

• defensive generation
• outcome indistinguishability
• online generative modeling
• unfalsifiable models
• multicalibration

Innovation

• Defensive Generation
• outcome indistinguishability
• multicalibration
• RKHS
• online learning