The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions

2026-02-25

AI Summary

This work addresses the limited accuracy and generalization capability of Deep Operator Networks (DeepONets) in scientific computing. Through error decomposition, the authors demonstrate that the branch network dominates the overall approximation error and exhibits spectral bias along with inter-mode parameter coupling. They further prove that the trunk network’s basis functions can be effectively replaced by the left singular vectors of the solution matrix. Building on this insight, they propose a novel architecture that constructs the trunk network using these singular vectors and reveal that modes corresponding to intermediate singular values contribute most significantly to the error. The study also shows that a shared-branch architecture outperforms independently stacked designs, offering both theoretical justification and a new perspective for enhancing DeepONet performance.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Operator learning has the potential to strongly impact scientific computing by learning solution operators for differential equations, potentially accelerating multi-query tasks such as design optimization and uncertainty quantification by orders of magnitude. Despite proven universal approximation properties, deep operator networks (DeepONets) often exhibit limited accuracy and generalization in practice, which hinders their adoption. Understanding these limitations is therefore crucial for further advancing the approach. This work analyzes performance limitations of the classical DeepONet architecture. It is shown that the approximation error is dominated by the branch network when the internal dimension is sufficiently large, and that the learned trunk basis can often be replaced by classical basis functions without a significant impact on performance. To investigate this further, a modified DeepONet is constructed in which the trunk network is replaced by the left singular vectors of the training solution matrix. This modification yields several key insights. First, a spectral bias in the branch network is observed, with coefficients of dominant, low-frequency modes learned more effectively. Second, due to singular-value scaling of the branch coefficients, the overall branch error is dominated by modes with intermediate singular values rather than the smallest ones. Third, using a shared branch network for all mode coefficients, as in the standard architecture, improves generalization of small modes compared to a stacked architecture in which coefficients are computed separately. Finally, strong and detrimental coupling between modes in parameter space is identified.

Problem

• DeepONet
• approximation error
• generalization
• operator learning
• spectral bias

Innovation

• DeepONet
• error decomposition
• spectral bias
• singular value decomposition
• mode coupling