🤖 AI Summary
Inverse problems are often ill-posed due to measurement ambiguities or system symmetries, limiting the uniqueness and fidelity of reconstructions when conventional diffusion models serve as generative priors. To address this, we propose Text Regularization (TReg), the first method to formulate natural language descriptions as differentiable regularization terms embedded into latent diffusion-based inverse solvers. TReg introduces a dynamic null-text optimization mechanism that adaptively strengthens or suppresses text priors during reverse sampling, enabling effective negative guidance. By integrating a CLIP text encoder with a latent diffusion model, TReg supports flexible text-conditioned reconstruction. Evaluated across diverse inverse problems—including super-resolution, deblurring, and phase retrieval—TReg consistently improves PSNR and SSIM by 2.1–4.3 dB on average, significantly enhancing both reconstruction uniqueness and perceptual quality.
📝 Abstract
The recent advent of diffusion models has led to significant progress in solving inverse problems, leveraging these models as effective generative priors. Nonetheless, there remain challenges related to the ill-posed nature of such problems, often due to inherent ambiguities in measurements or intrinsic system symmetries. To address this, drawing inspiration from the human ability to resolve visual ambiguities through perceptual biases, here we introduce a novel latent diffusion inverse solver by regularization by texts (TReg). Specifically, TReg applies the textual description of the preconception of the solution during the reverse diffusion sampling, of which the description is dynamically reinforced through null-text optimization for adaptive negation. Our comprehensive experimental results demonstrate that TReg successfully mitigates ambiguity in the inverse problems, enhancing their effectiveness and accuracy.
