Backward Filtering Forward Guiding

2025-05-23

AI Summary

This work addresses latent trajectory smoothing for stochastic processes on directed acyclic graphs (DAGs): inferring full posterior paths of hidden states across the entire graph given observations only at leaf nodes—a problem arising in phylogenetic inference, epidemic spread modeling, and structured signal processing. We propose BFFG (Backward Filtering–Forward Guiding), a novel paradigm that jointly couples backward information filtering with forward guided sampling. BFFG unifies treatment of discrete- and continuous-time models without requiring explicit transition densities. Its core components include backward likelihood potential construction, importance measure transformation, and weighted path sampling—naturally compatible with both MCMC and particle filtering. Evaluated on branching diffusion processes over tree-structured DAGs, BFFG achieves significantly improved posterior path estimation accuracy compared to standard methods. It extends probabilistic programming’s capacity to model structured stochastic dynamics, enabling scalable and flexible inference over complex graphical dependencies.

Technology Category

• Reasoning under Uncertainty: Probabilistic Programming
• Machine Learning: Probabilistic Circuits and Graphical Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Models for Web evolution

Abstract

We develop a general methodological framework for probabilistic inference in discrete- and continuous-time stochastic processes evolving on directed acyclic graphs (DAGs). The process is observed only at the leaf nodes, and the challenge is to infer its full latent trajectory: a smoothing problem that arises in fields such as phylogenetics, epidemiology, and signal processing. Our approach combines a backward information filtering step, which constructs likelihood-informed potentials from observations, with a forward guiding step, where a tractable process is simulated under a change of measure constructed from these potentials. This Backward Filtering Forward Guiding (BFFG) scheme yields weighted samples from the posterior distribution over latent paths and is amenable to integration with MCMC and particle filtering methods. We demonstrate that BFFG applies to both discrete- and continuous-time models, enabling probabilistic inference in settings where standard transition densities are intractable or unavailable. Our framework opens avenues for incorporating structured stochastic dynamics into probabilistic programming. We numerically illustrate our approach for a branching diffusion process on a directed tree.

Problem

• Infer latent trajectories in stochastic processes on DAGs
• Combine backward filtering and forward guiding for smoothing
• Enable inference when transition densities are intractable

Innovation

• Backward information filtering for likelihood-informed potentials
• Forward guiding under constructed change of measure
• Integration with MCMC and particle filtering methods