Active Diffusion Subsampling

2024-06-20

arXiv.org

AI Summary

This paper addresses high-dimensional signal reconstruction from sparse measurements by proposing an unsupervised active subsampling framework. Methodologically, it pioneers the integration of guided diffusion models with maximum-entropy active learning: during the reverse diffusion process, it dynamically models the Bayesian belief distribution and adaptively selects optimal measurement locations based on expected entropy maximization; it employs differentiable measurement modeling, eliminating task-specific fine-tuning and supporting arbitrary sampling rates. The key contributions are: (i) achieving unprecedented interpretability for black-box sampling strategies, and (ii) jointly optimizing sampling efficiency and reconstruction accuracy. On the fastMRI dataset, the method matches the reconstruction performance of supervised accelerated approaches and significantly outperforms fixed sampling strategies, thereby demonstrating the effectiveness and generalizability of unsupervised active sampling.

Technology Category

• Machine Learning: Active Learning
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Subsampling is commonly used to mitigate costs associated with data acquisition, such as time or energy requirements, motivating the development of algorithms for estimating the fully-sampled signal of interest $x$ from partially observed measurements $y$. In maximum-entropy sampling, one selects measurement locations that are expected to have the highest entropy, so as to minimize uncertainty about $x$. This approach relies on an accurate model of the posterior distribution over future measurements, given the measurements observed so far. Recently, diffusion models have been shown to produce high-quality posterior samples of high-dimensional signals using guided diffusion. In this work, we propose Active Diffusion Subsampling (ADS), a method for performing active subsampling using guided diffusion in which the model tracks a distribution of beliefs over the true state of $x$ throughout the reverse diffusion process, progressively decreasing its uncertainty by choosing to acquire measurements with maximum expected entropy, and ultimately generating the posterior distribution $p(x | y)$. ADS can be applied using pre-trained diffusion models for any subsampling rate, and does not require task-specific retraining - just the specification of a measurement model. Furthermore, the maximum entropy sampling policy employed by ADS is interpretable, enhancing transparency relative to existing methods using black-box policies. Experimentally, we show that ADS outperforms fixed sampling strategies, and study an application of ADS in Magnetic Resonance Imaging acceleration using the fastMRI dataset, finding that ADS performs competitively with supervised methods. Code available at https://active-diffusion-subsampling.github.io/.

Problem

• Estimating fully-sampled signals from partial measurements efficiently
• Designing subsampling masks using guided diffusion models
• Minimizing uncertainty via maximum entropy sampling policy

Innovation

• Active Diffusion Subsampling for intelligent mask design
• Uses guided diffusion to minimize measurement uncertainty
• Works with pre-trained models without task-specific retraining