ForwardFlow: Simulation only statistical inference using deep learning

2026-03-11

AI Summary

This study addresses the challenges of efficiency and accuracy in parameter estimation—a key inverse problem in complex statistical modeling where data are accessible only through simulation. The authors propose a frequentist approach based on a single summary network that directly learns parameter estimators from simulated data by minimizing the mean squared error between true parameters and the network’s output summaries. The method employs a branched network architecture with a collapse layer, theoretically designed to balance estimation accuracy under finite samples, robustness to contaminated data, and the ability to automatically approximate algorithmically reconstructed data. Experiments on genetic data simulations demonstrate that the framework successfully replicates the performance of the EM algorithm while exhibiting superior accuracy, robustness, and algorithmic approximation capability.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Deep learning models are being used for the analysis of parametric statistical models based on simulation-only frameworks. Bayesian models using normalizing flows simulate data from a prior distribution and are composed of two deep neural networks: a summary network that learns a sufficient statistic for the parameter and a normalizing flow that conditional on the summary network can approximate the posterior distribution. Here, we explore frequentist models that are based on a single summary network. During training, input of the network is a simulated data set based on a parameter and the loss function minimizes the mean-square error between learned summary and parameter. The network thereby solves the inverse problem of parameter estimation. We propose a branched network structure that contains collapsing layers that reduce a data set to summary statistics that are further mapped through fully connected layers to approximate the parameter estimate. We motivate our choice of network structure by theoretical considerations. In simulations we demonstrate three desirable properties of parameter estimates: finite sample exactness, robustness to data contamination, and algorithm approximation. These properties are achieved offering the the network varying sample size, contaminated data, and data needing algorithmic reconstruction during the training phase. In our simulations an EM-algorithm for genetic data is automatically approximated by the network. Simulation only approaches seem to offer practical advantages in complex modeling tasks where the simpler data simulation part is left to the researcher and the more complex problem of solving the inverse problem is left to the neural network. Challenging future work includes offering pre-trained models that can be used in a wide variety of applications.

Problem

• simulation-based inference
• parameter estimation
• inverse problem
• robustness
• finite sample

Innovation

• simulation-based inference
• deep learning
• summary network
• parameter estimation
• normalizing flows