Approximate Structured Diffusion for Sequence Labelling

2026-06-17

AI Summary

Traditional linear-chain conditional random fields (CRFs) are constrained by local dependency assumptions, limiting their ability to capture long-range label dependencies in sequences and thereby hindering tagging performance. This work proposes a novel approach that introduces a structured diffusion mechanism into sequence labeling for the first time. By training a neural parameterized CRF conditioned on noisy complete label sequences and integrating an approximate inference algorithm, the method effectively models global label structures. This strategy overcomes the locality limitation inherent in conventional CRFs and achieves a significant 16.5% reduction in error rate on part-of-speech tagging tasks, substantially improving overall tagging accuracy.

Abstract

Sequence labelling, a core task of Natural Language Processing (NLP), consists in assigning each token of an input sentence a label. From a Machine Learning point of view, sequence labelling is often cast as a Linear-Chain Conditional Random Field (CRF) parametrised by a neural network. While this approach gives good empirical results, CRFs assume a finite decision span (eg label bigrams) which can limit their expressivity and hurt performance when long-range dependencies are required. We show we can leverage diffusion to train a CRF conditioned on an entire label sequence, with the caveat that the condition is on a noisy version of labels. We show experimentally that this method, in conjunction with approximate CRF inference, improves label accuracy with a 16.5% error reduction for POS-tagging.

Problem

• sequence labelling
• Conditional Random Field
• long-range dependencies
• structured prediction
• Natural Language Processing

Innovation

• approximate structured diffusion
• sequence labelling
• conditional random field
• long-range dependencies
• noisy label conditioning