Invertible Fourier Neural Operators for Tackling Both Forward and Inverse Problems

2024-02-18

arXiv.org

AI Summary

Fourier Neural Operators (FNOs) struggle to jointly model forward prediction and inverse problem solving within a unified, invertible framework. Method: We propose the invertible Fourier Neural Operator (iFNO), featuring a novel invertible Fourier block that enables bidirectional latent-space mapping with parameter sharing; integration of a variational autoencoder (VAE) to explicitly model posterior distributions, thereby mitigating ill-posedness, data scarcity, and noise in inverse problems; and a three-stage collaborative training strategy. Contributions/Results: iFNO is the first FNO-based architecture enabling strict invertibility, and the first to deeply integrate VAEs with FNOs for uncertainty-aware inverse solving. Evaluated on seven forward/inverse benchmark tasks, iFNO consistently outperforms standard FNO and state-of-the-art inversion methods—achieving significant gains in accuracy, robustness, and data efficiency.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Learning to Search
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Fourier Neural Operator (FNO) is a powerful and popular operator learning method. However, FNO is mainly used in forward prediction, yet a great many applications rely on solving inverse problems. In this paper, we propose an invertible Fourier Neural Operator (iFNO) for jointly tackling the forward and inverse problems. We developed a series of invertible Fourier blocks in the latent channel space to share the model parameters, exchange the information, and mutually regularize the learning for the bi-directional tasks. We integrated a variational auto-encoder to capture the intrinsic structures within the input space and to enable posterior inference so as to mitigate challenges of illposedness, data shortage, noises that are common in inverse problems. We proposed a three-step process to combine the invertible blocks and the VAE component for effective training. The evaluations on seven benchmark forward and inverse tasks have demonstrated the advantages of our approach.

Problem

• Solving both forward and inverse problems using invertible Fourier Neural Operator
• Sharing model parameters and information for bi-directional tasks
• Mitigating illposedness and data shortage in inverse problems

Innovation

• Invertible Fourier blocks for bidirectional tasks
• Variational auto-encoder for posterior inference
• Three-step training combining invertible blocks and VAE