SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions

2026-04-24

AI Summary

This work addresses the limitation of traditional ReLU-based input convex neural networks (ICNNs), whose piecewise-linear structure struggles to effectively approximate smooth convex functions. To overcome this, the authors propose SOC-ICNN, which introduces second-order cone programming (SOCP) into the ICNN framework for the first time. By integrating positive semidefinite curvature with Euclidean norm cone primitives, SOC-ICNN constructs a natively smooth convex neural network. This approach strictly enhances the representational capacity of ICNNs while preserving the same forward computational complexity. Empirical results demonstrate that SOC-ICNN significantly outperforms conventional ICNNs in function approximation tasks and achieves competitive performance in downstream decision-making applications.

Technology Category

• Search and Optimization: Non-convex Optimization
• Computer Vision: Learning & Optimization for CV
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning
• Social Networks and Social Media: Computational social science

Abstract

Classical ReLU-based Input Convex Neural Networks (ICNNs) are equivalent to the optimal value functions of Linear Programming (LP). This intrinsic structural equivalence restricts their representational capacity to piecewise-linear polyhedral functions. To overcome this representational bottleneck, we propose the SOC-ICNN, an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP). By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native smooth curvature into the representation while preserving a rigorous optimization-theoretic interpretation. We formally prove that SOC-ICNNs strictly expand the representational space of ReLU-ICNNs without increasing the asymptotic order of forward-pass complexity. Extensive experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality. The code is available at https://github.com/Kanyooo/SOC-ICNN.

Problem

• Input Convex Neural Networks
• Representational Capacity
• Polyhedral Functions
• Second-Order Cone Programming
• Function Approximation

Innovation

• Second-Order Cone Programming
• Input Convex Neural Networks
• Conic Geometry
• Smooth Curvature
• Function Approximation