A Stochastic Finite-State Word-Segmentation Algorithm for Chinese
cmp-lg
2008-02-03 v2 Computation and Language
Abstract
We present a stochastic finite-state model for segmenting Chinese text into dictionary entries and productively derived words, and providing pronunciations for these words; the method incorporates a class-based model in its treatment of personal names. We also evaluate the system's performance, taking into account the fact that people often do not agree on a single segmentation.
Cite
@article{arxiv.cmp-lg/9405008,
title = {A Stochastic Finite-State Word-Segmentation Algorithm for Chinese},
author = {Richard Sproat and Chilin Shih and William Gale and Nancy Chang},
journal= {arXiv preprint arXiv:cmp-lg/9405008},
year = {2008}
}
Comments
To appear in Proceedings of ACL-94