Input convex neural networks as surrogates in mathematical optimisation

2026-08-10

AI Summary

This work addresses the computational challenges of embedding neural networks into mathematical optimization, where conventional feedforward neural networks (FNNs) yield mixed-integer programming (MIP) reformulations that are computationally expensive and suffer from loose relaxations. To overcome these limitations, the paper proposes using input convex neural networks (ICNNs) as surrogate models, leveraging their inherent convexity to construct tight linear programming (LP) relaxations. The authors establish, for the first time, an exact convex hull-based continuous relaxation of ICNNs over box domains, yielding an LP representation free of integrality gaps. Furthermore, they introduce a novel branch-and-bound algorithm that branches directly on input variables. Demonstrated across applications in humanitarian food aid allocation, oil well trajectory planning, and wine blending, the approach achieves approximation accuracy comparable to FNNs while significantly improving solution speed and scalability.

Abstract

Embedding trained neural networks as surrogates within optimisation problems is an established practice in operations research. The prevailing approach uses feedforward neural networks (FNNs) with ReLU activations, whose piecewise-linear structure admits an exact but computationally intensive mixed-integer programming (MIP) reformulation as the networks grow. We advocate input convex neural networks (ICNNs) as structurally superior surrogates when the underlying response is approximately convex or concave. The convex architecture offers two computational advantages. First, the ICNN-MIP formulation tends to yield a tighter linear programming (LP) relaxation than its FNN-MIP counterpart, with no integrality gap in favourable instances. Second, ICNNs uniquely admit an LP-based reformulation via epigraph representations of ReLU activations, though this embedding is not always exact. When it is not, we exploit the properties of ICNNs to construct the strongest continuous relaxation over box domains, namely, the convex hull of the ICNN's graph, bounded below by the epigraph and above by the concave envelope; this construction is tractable under input convexity but hard for general ReLU networks. On this basis, we develop a branch-and-bound algorithm that builds this relaxation at each node, branches directly on input variables rather than intermediate variables as in MIP reformulations, and terminates at the root node whenever the epigraph embedding is valid. Case studies on humanitarian food aid, oil well routing, and wine blending show that ICNN surrogates match FNN accuracy and deliver gains in solve time and scalability, supporting ICNN as the default surrogate when the underlying function is convex, concave, or well-approximated as such.

Problem

• input convex neural networks
• surrogate modeling
• mathematical optimization
• mixed-integer programming
• convexity

Innovation

• Input Convex Neural Networks
• Mixed-Integer Programming
• Convex Relaxation
• Epigraph Representation
• Branch-and-Bound