Hyper Input Convex Neural Networks for Shape Constrained Learning and Optimal Transport

2026-04-29

AI Summary

This work addresses the limited parameter efficiency and poor scalability of existing Input Convex Neural Networks (ICNNs) in shape-constrained learning and high-dimensional optimal transport. To overcome these limitations, the authors propose Hyper Input Convex Neural Networks (HyCNNs), which integrate the Maxout activation mechanism into the ICNN architecture. This design strictly preserves input convexity while substantially enhancing model expressivity and scalability. Theoretical analysis demonstrates that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions. Empirical evaluations on convex regression, interpolation, and high-dimensional optimal transport tasks using single-cell RNA sequencing data show that HyCNNs significantly outperform baseline methods, including standard ICNNs and multilayer perceptrons (MLPs).

Abstract

We introduce Hyper Input Convex Neural Networks (HyCNNs), a novel neural network architecture designed for learning convex functions. HyCNNs combine the principles of Maxout networks with input convex neural networks (ICNNs) to create a neural network that is always convex in the input, theoretically capable of leveraging depth, and performs reliable when trained at scale compared to ICNNs. Concretely, we prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions up to a given precision. Throughout a series of synthetic experiments, we demonstrate that HyCNNs outperform existing ICNNs and MLPs in terms of predictive performance for convex regression and interpolation tasks. We further apply HyCNNs to learn high-dimensional optimal transport maps for synthetic examples and for single-cell RNA sequencing data, where they oftentimes outperform ICNN-based neural optimal transport methods and other baselines across a wide range of settings.

Problem

• convex neural networks
• optimal transport
• shape constrained learning
• input convexity
• high-dimensional function approximation

Innovation

• Hyper Input Convex Neural Networks
• Input Convex Neural Networks
• Optimal Transport
• Convex Function Approximation
• Maxout Networks