Latent Target Score Matching, with an application to Simulation-Based Inference

2026-02-06

AI Summary

This work addresses the challenge of score estimation in the presence of latent variables, where conventional denoising score matching (DSM) suffers from high variance at low noise levels, and target score matching (TSM) is inapplicable due to the unavailability of clean-data scores. To overcome this limitation, the authors propose Latent-variable Target Score Matching (LTSM), which extends TSM to settings with latent variables for the first time. LTSM leverages the score of the joint distribution to provide low-variance supervision for the marginal score and integrates DSM into a hybrid training strategy that ensures robustness across varying noise scales. Experimental results demonstrate that LTSM substantially reduces estimation variance, leading to improved score estimation accuracy and enhanced generative sample quality.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Denoising score matching (DSM) for training diffusion models may suffer from high variance at low noise levels. Target Score Matching (TSM) mitigates this when clean data scores are available, providing a low-variance objective. In many applications clean scores are inaccessible due to the presence of latent variables, leaving only joint signals exposed. We propose Latent Target Score Matching (LTSM), an extension of TSM to leverage joint scores for low-variance supervision of the marginal score. While LTSM is effective at low noise levels, a mixture with DSM ensures robustness across noise scales. Across simulation-based inference tasks, LTSM consistently improves variance, score accuracy, and sample quality.

Problem

• simulation-based inference
• latent variables
• score matching
• diffusion models
• high variance

Innovation

• Latent Target Score Matching
• Score Matching
• Simulation-Based Inference
• Diffusion Models
• Low-Variance Estimation