Bregman Conditional Random Fields: Sequence Labeling with Parallelizable Inference Algorithms

2025-05-31

AI Summary

To address the slow inference speed and limited parallelizability of traditional linear-chain CRFs in sequence labeling, this paper proposes the Bregman Conditional Random Field (BCRF). BCRF is the first structured prediction model to incorporate iterative Bregman projections, enabling fully parallel graphical inference. Integrated with the Fenchel–Young loss, it supports end-to-end differentiable training and partial-label learning. Empirically, BCRF achieves several-fold inference speedup over standard CRFs while maintaining comparable accuracy; it also outperforms parallel baselines—such as mean-field approximation—under resource-constrained settings. The core contributions are: (i) a novel parallelizable structured inference framework grounded in Bregman divergence; and (ii) a differentiable projection-based iteration mechanism jointly optimized with structured output prediction. This paradigm bridges efficient parallel computation and principled probabilistic modeling for structured prediction.

Technology Category

• Machine Learning: Structured Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We propose a novel discriminative model for sequence labeling called Bregman conditional random fields (BCRF). Contrary to standard linear-chain conditional random fields, BCRF allows fast parallelizable inference algorithms based on iterative Bregman projections. We show how such models can be learned using Fenchel-Young losses, including extension for learning from partial labels. Experimentally, our approach delivers comparable results to CRF while being faster, and achieves better results in highly constrained settings compared to mean field, another parallelizable alternative.

Problem

• Proposes BCRF for faster parallel sequence labeling
• Enables efficient inference via Bregman projections
• Outperforms CRF and mean field in constrained settings

Innovation

• Bregman CRF enables parallelizable inference algorithms
• Uses Fenchel-Young losses for model learning
• Faster than CRF with comparable accuracy