Pseudo-Invertible Neural Networks

2026-02-05

AI Summary

This work addresses zero-shot inverse problems under nonlinear degradation by formally introducing the concept of a nonlinear pseudoinverse and its geometric properties. It proposes the Surjective Pseudoinvertible Neural Network (SPNN) to explicitly model the nonlinear pseudoinverse. By integrating Nonlinear Back-Projection (NLBP) with null-space projection techniques from diffusion models, SPNN enables high-fidelity inversion and semantically controllable generation for arbitrary nonlinear degradations—such as optical distortions or semantic abstractions—without requiring retraining. This approach successfully extends zero-shot inverse problem solving from linear systems to nonlinear settings, significantly enhancing both generation consistency and controllability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Non-convex Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

The Moore-Penrose Pseudo-inverse (PInv) serves as the fundamental solution for linear systems. In this paper, we propose a natural generalization of PInv to the nonlinear regime in general and to neural networks in particular. We introduce Surjective Pseudo-invertible Neural Networks (SPNN), a class of architectures explicitly designed to admit a tractable non-linear PInv. The proposed non-linear PInv and its implementation in SPNN satisfy fundamental geometric properties. One such property is null-space projection or"Back-Projection", $x'= x + A^\dagger(y-Ax)$, which moves a sample $x$ to its closest consistent state $x'$ satisfying $Ax=y$. We formalize Non-Linear Back-Projection (NLBP), a method that guarantees the same consistency constraint for non-linear mappings $f(x)=y$ via our defined PInv. We leverage SPNNs to expand the scope of zero-shot inverse problems. Diffusion-based null-space projection has revolutionized zero-shot solving for linear inverse problems by exploiting closed-form back-projection. We extend this method to non-linear degradations. Here,"degradation"is broadly generalized to include any non-linear loss of information, spanning from optical distortions to semantic abstractions like classification. This approach enables zero-shot inversion of complex degradations and allows precise semantic control over generative outputs without retraining the diffusion prior.

Problem

• Pseudo-inverse
• Non-linear inverse problems
• Zero-shot inversion
• Neural networks
• Degradation modeling

Innovation

• Pseudo-Invertible Neural Networks
• Non-Linear Back-Projection
• Zero-Shot Inverse Problems
• Diffusion Models
• Moore-Penrose Pseudo-inverse