English

iNLTK: Natural Language Toolkit for Indic Languages

Computation and Language 2021-02-15 v2

Abstract

We present iNLTK, an open-source NLP library consisting of pre-trained language models and out-of-the-box support for Data Augmentation, Textual Similarity, Sentence Embeddings, Word Embeddings, Tokenization and Text Generation in 13 Indic Languages. By using pre-trained models from iNLTK for text classification on publicly available datasets, we significantly outperform previously reported results. On these datasets, we also show that by using pre-trained models and data augmentation from iNLTK, we can achieve more than 95% of the previous best performance by using less than 10% of the training data. iNLTK is already being widely used by the community and has 40,000+ downloads, 600+ stars and 100+ forks on GitHub. The library is available at https://github.com/goru001/inltk.

Keywords

Cite

@article{arxiv.2009.12534,
  title  = {iNLTK: Natural Language Toolkit for Indic Languages},
  author = {Gaurav Arora},
  journal= {arXiv preprint arXiv:2009.12534},
  year   = {2021}
}

Comments

Accepted at EMNLP2020's NLP-OSS workshop

R2 v1 2026-06-23T18:48:43.234Z