Search papers, labs, and topics across Lattice.
This paper introduces an innovative approach to sequence labeling by leveraging diffusion to train a Conditional Random Field (CRF) that conditions on an entire label sequence rather than just local dependencies. This method addresses the limitations of traditional CRFs, which struggle with long-range dependencies due to their finite decision span. Experimental results demonstrate a significant improvement in label accuracy, achieving a 16.5% reduction in error for part-of-speech tagging tasks.
Training CRFs with diffusion allows for capturing long-range dependencies, resulting in a 16.5% reduction in labeling errors for POS-tagging.
Sequence labelling, a core task of Natural Language Processing (NLP), consists in assigning each token of an input sentence a label. From a Machine Learning point of view, sequence labelling is often cast as a Linear-Chain Conditional Random Field (CRF) parametrised by a neural network. While this approach gives good empirical results, CRFs assume a finite decision span (eg label bigrams) which can limit their expressivity and hurt performance when long-range dependencies are required. We show we can leverage diffusion to train a CRF conditioned on an entire label sequence, with the caveat that the condition is on a noisy version of labels. We show experimentally that this method, in conjunction with approximate CRF inference, improves label accuracy with a 16.5% error reduction for POS-tagging.
