Dimension-reduced Reconstruction Map Learning for Parameter Estimation in Likelihood-Free Inference Problems

2024-07-19

Journal of Machine Learning for Modeling and Computing

AI Summary

For simulator-based models lacking closed-form likelihoods, existing neural reconstruction-based parameter estimation methods suffer severely from the curse of dimensionality under high-dimensional observations, leading to substantial accuracy degradation. This paper proposes a domain-knowledge-integrated dimensionality reduction framework for learning reconstruction mappings: it jointly models prior-driven nonlinear dimensionality reduction (e.g., manifold projection or feature selection) with neural reconstruction mappings, theoretically balancing dimensionality-reduction distortion against function approximation error. The method requires no likelihood evaluation and enables likelihood-free inference via training on synthetic data. On multiple benchmark tasks, it reduces estimation error by 32%–57% compared to standard reconstruction mapping, approximate Bayesian computation (ABC), and synthetic likelihood approaches, while demonstrating markedly improved robustness to high-dimensional observations.

Technology Category

• Machine Learning: Learning with Manifolds
• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: User model development and evaluation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Many application areas rely on models that can be readily simulated but lack a closed-form likelihood, or an accurate approximation under arbitrary parameter values. Existing parameter estimation approaches in this setting are generally approximate. Recent work on using neural network models to reconstruct the mapping from the data space to the parameters from a set of synthetic parameter-data pairs suffers from the curse of dimensionality, resulting in inaccurate estimation as the data size grows. We propose a dimension-reduced approach to likelihood-free estimation which combines the ideas of reconstruction map estimation with dimension-reduction approaches based on subject-specific knowledge. We examine the properties of reconstruction map estimation with and without dimension reduction and explore the trade-off between approximation error due to information loss from reducing the data dimension and approximation error. Numerical examples show that the proposed approach compares favorably with reconstruction map estimation, approximate Bayesian computation, and synthetic likelihood estimation.

Problem

• Estimating parameters in models without closed-form likelihoods
• Addressing dimensionality curse in neural network-based reconstruction maps
• Balancing information loss and approximation error in dimension reduction

Innovation

• Dimension-reduced reconstruction map learning
• Combines neural networks with dimension-reduction
• Improves accuracy in likelihood-free estimation