Enhancing Neural Autoregressive Distribution Estimators for Image Reconstruction

2025-06-03

AI Summary

This paper addresses image reconstruction from sparse pixel observations—i.e., predicting full images given only a small subset of (random or low-discrepancy) sampled pixel blocks. We propose a generalized Convolutional Neural Autoregressive Distribution Estimator (ConvNADE) capable of modeling both real-valued and RGB images. Our key contributions are twofold: first, we introduce quasi-Monte Carlo–inspired low-discrepancy sequences for pixel sampling—the first such application in autoregressive image modeling—yielding substantially improved sampling efficiency and generalization; second, we redesign the network architecture to support continuous-value modeling and joint estimation across color channels. Evaluated on standard benchmarks, our method achieves significantly lower test negative log-likelihood compared to prior approaches, while producing reconstructions with more coherent global structure and more photorealistic textures. These results empirically validate the effectiveness of jointly optimizing low-discrepancy sampling and generalized autoregressive modeling for sparse-image reconstruction.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Non-convex Optimization

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Autoregressive models are often employed to learn distributions of image data by decomposing the $D$-dimensional density function into a product of one-dimensional conditional distributions. Each conditional depends on preceding variables (pixels, in the case of image data), making the order in which variables are processed fundamental to the model performance. In this paper, we study the problem of observing a small subset of image pixels (referred to as a pixel patch) to predict the unobserved parts of the image. As our prediction mechanism, we propose a generalized and computationally efficient version of the convolutional neural autoregressive distribution estimator (ConvNADE) model adapted for real-valued and color images. Moreover, we investigate the quality of image reconstruction when observing both random pixel patches and low-discrepancy pixel patches inspired by quasi-Monte Carlo theory. Experiments on benchmark datasets demonstrate that choosing the pixels akin to a low-discrepancy sequence reduces test loss and produces more realistic reconstructed images.

Problem

• Improving autoregressive models for image reconstruction
• Predicting unobserved image parts from small pixel subsets
• Enhancing reconstruction quality using low-discrepancy pixel patches

Innovation

• Generalized ConvNADE for real-valued images
• Low-discrepancy pixel patches for reconstruction
• Efficient autoregressive conditional distribution modeling