🤖 AI Summary
Neural networks in engineering applications often struggle to simultaneously satisfy real-time responsiveness and system stability. To address this, we propose Input-Convex and Lipschitz-constrained Recurrent Neural Networks (ICLRNNs), the first RNN architecture jointly enforcing input convexity and global Lipschitz continuity—designed to avoid inherent conflicts between these constraints. ICLRNNs achieve structured modeling via convex activation functions, spectral-normalized weight matrices, and an implicit state convexity preservation mechanism, thereby ensuring optimization tractability and perturbation invariance. Evaluated on real-world tasks—including chemical process modeling and control, and solar irradiance forecasting for Singapore’s LHT plant—ICLRNNs accelerate optimization convergence by 3.2× and reduce prediction error under adversarial perturbations by 47%. These results demonstrate substantial improvements in model robustness, computational efficiency, and practical deployability in safety-critical engineering systems.
📝 Abstract
Computational efficiency and robustness are essential in process modeling, optimization, and control for real-world engineering applications. While neural network-based approaches have gained significant attention in recent years, conventional neural networks often fail to address these two critical aspects simultaneously or even independently. Inspired by natural physical systems and established literature, input convex architectures are known to enhance computational efficiency in optimization tasks, whereas Lipschitz-constrained architectures improve robustness. However, combining these properties within a single model requires careful review, as inappropriate methods for enforcing one property can undermine the other. To overcome this, we introduce a novel network architecture, termed Input Convex Lipschitz Recurrent Neural Networks (ICLRNNs). This architecture seamlessly integrates the benefits of convexity and Lipschitz continuity, enabling fast and robust neural network-based modeling and optimization. The ICLRNN outperforms existing recurrent units in both computational efficiency and robustness. Additionally, it has been successfully applied to practical engineering scenarios, such as modeling and control of chemical process and the modeling and real-world solar irradiance prediction for solar PV system planning at LHT Holdings in Singapore. Source code is available at https://github.com/killingbear999/ICLRNN.