On the role of memorization in learned priors for geophysical inverse problems

2026-03-20

AI Summary

This work addresses the issue of sample memorization in deep generative priors for geophysical inversion, which arises due to scarce training data. The study reveals that the resulting posterior is equivalent to a likelihood-weighted empirical distribution over the training samples and, for the first time, derives closed-form Gaussian mixture priors and posteriors for diffusion models. By locally linearizing the forward operator and incorporating Jacobian analysis, the authors construct a Gaussian mixture posterior endowed with local geometric constraints, offering a novel theoretical perspective on the generalization capability of data-driven inversion methods. Experiments on both simplified inversion and full-waveform inversion validate the theoretical predictions regarding memorization effects and demonstrate their tangible impact on reconstruction quality.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Learned priors based on deep generative models offer data-driven regularization for seismic inversion, but training them requires a dataset of representative subsurface models -- a resource that is inherently scarce in geoscience applications. Since the training objective of most generative models can be cast as maximum likelihood on a finite dataset, any such model risks converging to the empirical distribution -- effectively memorizing the training examples rather than learning the underlying geological distribution. We show that the posterior under such a memorized prior reduces to a reweighted empirical distribution -- i.e., a likelihood-weighted lookup among the stored training examples. For diffusion models specifically, memorization yields a Gaussian mixture prior in closed form, and linearizing the forward operator around each training example gives a Gaussian mixture posterior whose components have widths and shifts governed by the local Jacobian. We validate these predictions on a stylized inverse problem and demonstrate the consequences of memorization through diffusion posterior sampling for full waveform inversion.

Problem

• memorization
• learned priors
• geophysical inverse problems
• generative models
• seismic inversion

Innovation

• memorization
• learned priors
• diffusion models
• geophysical inverse problems
• Gaussian mixture posterior