Learning Single Index Models with Diffusion Priors

2025-05-27

AI Summary

This paper addresses signal recovery in semiparametric single-index models (SIMs) with discontinuous or unknown link functions. We propose an efficient reconstruction method grounded in diffusion models (DMs), which eliminates the conventional reliance on continuity assumptions or prior knowledge of the link function. To our knowledge, this is the first work to systematically incorporate diffusion priors into solving SIM inverse problems; reconstruction is achieved via a single unconditional sampling run followed by only a partial reverse-diffusion process. Theoretical analysis establishes consistency and stability of the reconstructed solution. Extensive experiments demonstrate that our method achieves significantly higher image reconstruction accuracy than state-of-the-art approaches under diverse nonlinear (including discontinuous) measurement settings, while drastically reducing the number of neural network function evaluations—thereby attaining both high fidelity and computational efficiency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Diffusion models (DMs) have demonstrated remarkable ability to generate diverse and high-quality images by efficiently modeling complex data distributions. They have also been explored as powerful generative priors for signal recovery, resulting in a substantial improvement in the quality of reconstructed signals. However, existing research on signal recovery with diffusion models either focuses on specific reconstruction problems or is unable to handle nonlinear measurement models with discontinuous or unknown link functions. In this work, we focus on using DMs to achieve accurate recovery from semi-parametric single index models, which encompass a variety of popular nonlinear models that may have {em discontinuous} and {em unknown} link functions. We propose an efficient reconstruction method that only requires one round of unconditional sampling and (partial) inversion of DMs. Theoretical analysis on the effectiveness of the proposed methods has been established under appropriate conditions. We perform numerical experiments on image datasets for different nonlinear measurement models. We observe that compared to competing methods, our approach can yield more accurate reconstructions while utilizing significantly fewer neural function evaluations.

Problem

• Recover signals from nonlinear single index models
• Handle discontinuous or unknown link functions
• Improve accuracy with fewer neural evaluations

Innovation

• Uses diffusion models for signal recovery
• Handles nonlinear models with unknown links
• Requires one unconditional sampling round