Tuned and GPU-accelerated parallel data mining from comparable corpora
Abstract
The multilingual nature of the world makes translation a crucial requirement today. Parallel dictionaries constructed by humans are a widely-available resource, but they are limited and do not provide enough coverage for good quality translation purposes, due to out-of-vocabulary words and neologisms. This motivates the use of statistical translation systems, which are unfortunately dependent on the quantity and quality of training data. Such has a very limited availability especially for some languages and very narrow text domains. Is this research we present our improvements to Yalign mining methodology by reimplementing the comparison algorithm, introducing a tuning scripts and by improving performance using GPU computing acceleration. The experiments are conducted on various text domains and bi-data is extracted from the Wikipedia dumps.
Cite
@article{arxiv.1509.08639,
title = {Tuned and GPU-accelerated parallel data mining from comparable corpora},
author = {Krzysztof Wołk and Krzysztof Marasek},
journal= {arXiv preprint arXiv:1509.08639},
year = {2015}
}
Comments
Machine translation, comparable corpora, Machine learning, NLP, Knowledge-free learning, Unsupervised bi-lingual data mining