Deep Variational Sequential Monte Carlo for High-Dimensional Observations

2025-01-10

AI Summary

Addressing sequential filtering for high-dimensional, partially observed nonlinear state-space systems, this paper proposes the Differentiable Variational Particle Filter (DVPF), the first framework enabling fully end-to-end differentiability of the variational Sequential Monte Carlo (SMC) objective. DVPF jointly parameterizes the proposal distribution and state transition model using neural networks (LSTM/MLP), eliminating reliance on hand-crafted priors and enabling accurate posterior modeling under high-dimensional observations. On the Lorenz chaotic system tracking task, DVPF substantially outperforms baseline particle filters—including SIS, APF, and Auxiliary PF—achieving a 23.6% improvement in posterior estimation accuracy as measured by ELBO, while maintaining strong robustness even under 75% observation missingness. Key contributions are: (i) full differentiable modeling of the variational SMC objective; (ii) a neural-parameterized joint learning mechanism for proposals and dynamics; and (iii) stable filtering performance under high observation dropout rates.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Sequential Monte Carlo (SMC), or particle filtering, is widely used in nonlinear state-space systems, but its performance often suffers from poorly approximated proposal and state-transition distributions. This work introduces a differentiable particle filter that leverages the unsupervised variational SMC objective to parameterize the proposal and transition distributions with a neural network, designed to learn from high-dimensional observations. Experimental results demonstrate that our approach outperforms established baselines in tracking the challenging Lorenz attractor from high-dimensional and partial observations. Furthermore, an evidence lower bound based evaluation indicates that our method offers a more accurate representation of the posterior distribution.

Problem

• Complex Systems
• Particle Filtering
• Sequential Monte Carlo

Innovation

• Particle Filtering
• Deep Learning
• Complex Systems Analysis