Path-dependent Discrete Amortized Inference

2026-08-09

AI Summary

This work addresses the challenge of efficiently sampling combinatorial discrete objects from unnormalized posterior distributions, where existing amortized inference methods based on Markov decision processes suffer from state aliasing, leading to impaired signal propagation and limited expressivity. To overcome these limitations, the authors propose a path-dependent amortized sampling framework that introduces a learnable implicit dynamical system, enabling the policy to model the full generation trajectory rather than relying solely on the current state. This approach effectively relaxes the Markov assumption and allows for conditional modeling over entire trajectories. Theoretically, the framework preserves the scalability of existing discrete amortized algorithms under this extended setting. Empirical results demonstrate that the proposed method significantly accelerates training convergence and enhances exploration in the state space, outperforming current approaches on standard benchmark tasks.

Abstract

We consider the problem of sampling compositional and discrete objects from a given unnormalized posterior distribution. Notably, recent studies have shown that this problem can be efficiently solved by learning a deterministic Markov Decision Process (MDP) that progressively builds each object in proportion to the posterior. In this work, however, we demonstrate that the Markovian assumption can both hamper signal propagation during training and catastrophically reduce the learned sampler's expressivity due to state aliasing. To address these issues, we propose lifting the MDP with a learnable latent dynamical system that allows the underlying policy to depend on the entire past trajectory---and not only on the current state. In view of this, we refer to the resulting method as path-dependent discrete amortized inference. Importantly, we provably extend existing learning algorithms for discrete amortized samplers to our setting. In experiments on standard benchmark problems, we also show that our approach often leads to faster learning convergence and improved state space exploration relatively to prior techniques.

Problem

• discrete sampling
• posterior inference
• Markov assumption
• state aliasing
• compositional objects

Innovation

• path-dependent inference
• discrete amortized inference
• latent dynamical system
• non-Markovian policy
• state aliasing