Saddle Networks: Structure-Preserving Architectures for Convex-Concave Functions

2026-05-27

AI Summary

This work addresses the challenge of modeling convex-concave structures in minimax problems by proposing a structured separable decomposition approach. By imposing sign and concavity constraints on the output layer of neural networks, the method inherently preserves convexity in variable \(x\) and concavity in variable \(y\). The study establishes, for the first time, a one-dimensional universal approximation theorem under mixed Monge-type convexity conditions, which directly informs the design of a neural network architecture that strictly maintains the underlying convex-concave geometry. Combining convexity-preserving networks with a simple output transformation, the approach achieves high-accuracy approximation of smooth, nonsmooth, and high-rank convex-concave functions, demonstrating superior performance on both one-dimensional and five-dimensional benchmark problems.

Abstract

Saddle-point models arise throughout optimization, optimal transport, robust learning, and control. In many applications, the relevant function f(x,y) is convex in x and concave in y, and preserving this geometry is essential for obtaining tractable min--max formulations and reliable certificates. We introduce a structured separable decomposition that preserves the convex-concave geometry and prove a complete one-dimensional approximation theorem under a mixed Monge-type convexity condition. We then describe practical saddle network architectures that preserve convexity in x and concavity in y by construction. The proposed architectures require only convexity-preserving neural networks, together with simple output transformations enforcing sign and concavity constraints. Finally, we report numerical benchmarks in dimension 1 and 5, showing that the proposed saddle networks achieve high accuracy on smooth, nonsmooth, and high-rank convex--concave test functions.

Problem

• convex-concave functions
• saddle-point models
• structure-preserving
• min-max optimization
• geometric structure

Innovation

• saddle networks
• convex-concave functions
• structure-preserving architecture
• Monge-type convexity
• min-max optimization