Local Inconsistency Resolution: The Interplay between Attention and Control in Probabilistic Models

2026-04-18

AI Summary

This work addresses the inefficiency in inference and learning caused by locally inconsistent beliefs in probabilistic models by proposing the Local Inconsistency Resolution (LIR) framework. LIR iteratively focuses on critical substructures within probabilistic dependency graphs (PDGs), dynamically resolving local inconsistencies through an attention mechanism coupled with controllable parameter adjustments. The framework unifies and generalizes several existing algorithms—including EM, belief propagation, adversarial training, GANs, and GFlowNets—and introduces a more natural loss function for GFlowNets to accelerate convergence. Experimental results on synthetic discrete PDG tasks demonstrate that LIR converges significantly faster than global optimization approaches, with particularly pronounced performance gains in GFlowNet-based settings.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Local Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We present a generic algorithm for learning and approximate inference with an intuitive epistemic interpretation: iteratively focus on a subset of the model and resolve inconsistencies using the parameters under control. This framework, which we call Local Inconsistency Resolution (LIR) is built upon Probabilistic Dependency Graphs (PDGs), which provide a flexible representational foundation capable of capturing inconsistent beliefs. We show how LIR unifies and generalizes a wide variety of important algorithms in the literature, including the Expectation-Maximization (EM) algorithm, belief propagation, adversarial training, GANs, and GFlowNets. In the last case, LIR actually suggests a more natural loss, which we demonstrate improves GFlowNet convergence. Each method can be recovered as a specific instance of LIR by choosing a procedure to direct focus (attention and control). We implement this algorithm for discrete PDGs and study its properties on synthetically generated PDGs, comparing its behavior to the global optimization semantics of the full PDG.

Problem

• Local Inconsistency Resolution
• Probabilistic Models
• Inconsistent Beliefs
• Approximate Inference
• Attention and Control

Innovation

• Local Inconsistency Resolution
• Probabilistic Dependency Graphs
• attention and control
• unified inference framework
• GFlowNets