Automatic Speech Recognition (ASR) generates text which is most of the times devoid of any punctuation. Absence of punctuation is text can affect readability. Also, down stream NLP tasks such as sentiment analysis, machine translation, greatly benefit by having punctuation and sentence boundary information. We present an approach for automatic punctuation of text using a pretrained IndicBERT model. Inverse text normalization is done by hand writing weighted finite state transducer (WFST) grammars. We have developed this tool for 11 Indic languages namely Hindi, Tamil, Telugu, Kannada, Gujarati, Marathi, Odia, Bengali, Assamese, Malayalam and Punjabi. All code and data is publicly. available
Cite
@article{arxiv.2203.16825,
title = {indic-punct: An automatic punctuation restoration and inverse text normalization framework for Indic languages},
author = {Anirudh Gupta and Neeraj Chhimwal and Ankur Dhuriya and Rishabh Gaur and Priyanshi Shah and Harveen Singh Chadha and Vivek Raghavan},
journal= {arXiv preprint arXiv:2203.16825},
year = {2022}
}
Comments
Submitted to InterSpeech 2022. arXiv admin note: text overlap with arXiv:2104.05055 by other authors