NeuroLifting: Neural Inference on Markov Random Fields at Scale

2024-11-28

arXiv.org

AI Summary

To address the trade-off between efficiency and accuracy in large-scale Markov Random Field (MRF) inference, this paper proposes a differentiable reparameterization method based on Graph Neural Networks (GNNs), transforming discrete optimization into a parallelizable gradient-descent problem. The core innovation lies in the first integration of the lifting technique into a non-parametric GNN framework, which smooths the loss landscape via neural parameterization to enable efficient and scalable approximate inference. This approach breaks the computational complexity bottlenecks of conventional algorithms, achieving linear time complexity in graph size. Experiments demonstrate that, on medium-scale MRFs, the method attains solution quality comparable to the exact solver Toulbar2; on large-scale instances, it consistently outperforms all baseline methods—including both classical and learning-based approaches—in both solution quality and runtime efficiency.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Inference in large-scale Markov Random Fields (MRFs) is a critical yet challenging task, traditionally approached through approximate methods like belief propagation and mean field, or exact methods such as the Toulbar2 solver. These strategies often fail to strike an optimal balance between efficiency and solution quality, particularly as the problem scale increases. This paper introduces NeuroLifting, a novel technique that leverages Graph Neural Networks (GNNs) to reparameterize decision variables in MRFs, facilitating the use of standard gradient descent optimization. By extending traditional lifting techniques into a non-parametric neural network framework, NeuroLifting benefits from the smooth loss landscape of neural networks, enabling efficient and parallelizable optimization. Empirical results demonstrate that, on moderate scales, NeuroLifting performs very close to the exact solver Toulbar2 in terms of solution quality, significantly surpassing existing approximate methods. Notably, on large-scale MRFs, NeuroLifting delivers superior solution quality against all baselines, as well as exhibiting linear computational complexity growth. This work presents a significant advancement in MRF inference, offering a scalable and effective solution for large-scale problems.

Problem

• Balancing efficiency and quality in large-scale MRF inference
• Reparameterizing MRF variables using GNNs for gradient descent
• Achieving scalable MRF inference with linear complexity growth

Innovation

• Uses GNNs to reparameterize MRF decision variables
• Extends lifting techniques with neural networks
• Enables gradient descent optimization for MRFs