Search papers, labs, and topics across Lattice.
This paper tackles the problem of Chinese word boundary recovery in noisy text by framing it as an alignment-based projection task, where word boundaries from a cleaner target sentence are projected onto a noisy source sentence via character alignment. They introduce two new evaluation datasets: a manually checked learner Chinese benchmark and a synthetic benchmark derived from the Chinese Penn Treebank. Experiments demonstrate that this projection method effectively corrects over-segmentation errors common in learner input, outperforming direct segmentation approaches.
Alignment-based projection offers a surprisingly effective way to fix broken Chinese word boundaries in noisy text, outperforming direct segmentation and stabilizing annotation pipelines.
Chinese word segmentation is especially fragile in non-standard text, where language learner errors and other character-level divergences disrupt the word boundaries assumed by downstream annotation and evaluation. This paper formulates Chinese word boundary recovery as an alignment-based projection task. Given a noisy source sentence and a cleaner target counterpart, we first align the two strings at the character level and then project target-side word boundaries back onto the source. Beyond the recovery method itself, we introduce two evaluation resources: a manually checked learner Chinese benchmark based on MuCGEC and a controlled synthetic benchmark derived from the Chinese Penn Treebank. Experiments show that direct segmentation remains vulnerable to compound fragmentation in learner input, whereas the proposed two step projection method corrects many over-segmentation errors by using the corrected target to recover source-side word spans. The results show that word boundary recovery is distinct from ordinary segmentation and that alignment projection provides a principled mechanism for stabilizing Chinese annotation and evaluation under noisy input.
