Self-Composing Neural Operators with Depth and Accuracy Scaling via Adaptive Train-and-Unroll Approach

2025-08-28

AI Summary

In large-scale scientific machine learning, neural operators often struggle to simultaneously achieve high efficiency and accuracy. Method: This paper proposes the Self-Composing Neural Operator (SCNO) framework, which employs iterative deep unfolding of a single operator block, integrating a multigrid-inspired backbone network with an adaptive training strategy featuring progressive depth growth. SCNO explicitly leverages a scaling law governing accuracy versus depth to dynamically co-optimize model capacity and computational resources. Contributions/Results: SCNO achieves state-of-the-art performance on standard PDE-solving benchmarks. Moreover, in high-frequency ultrasound computed tomography, it significantly improves modeling accuracy and generalization for wave propagation phenomena—demonstrating its effectiveness and scalability for high-oscillation physical modeling.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

In this work, we propose a novel framework to enhance the efficiency and accuracy of neural operators through self-composition, offering both theoretical guarantees and practical benefits. Inspired by iterative methods in solving numerical partial differential equations (PDEs), we design a specific neural operator by repeatedly applying a single neural operator block, we progressively deepen the model without explicitly adding new blocks, improving the model's capacity. To train these models efficiently, we introduce an adaptive train-and-unroll approach, where the depth of the neural operator is gradually increased during training. This approach reveals an accuracy scaling law with model depth and offers significant computational savings through our adaptive training strategy. Our architecture achieves state-of-the-art (SOTA) performance on standard benchmarks. We further demonstrate its efficacy on a challenging high-frequency ultrasound computed tomography (USCT) problem, where a multigrid-inspired backbone enables superior performance in resolving complex wave phenomena. The proposed framework provides a computationally tractable, accurate, and scalable solution for large-scale data-driven scientific machine learning applications.

Problem

• Enhancing neural operator efficiency and accuracy via self-composition
• Training progressively deepening models with adaptive train-and-unroll approach
• Solving large-scale data-driven scientific machine learning applications

Innovation

• Self-composing neural operators via iterative block application
• Adaptive train-and-unroll approach for depth scaling
• Multigrid-inspired backbone for complex wave resolution