English

IruMozhi: Automatically classifying diglossia in Tamil

Computation and Language 2023-11-15 v1

Abstract

Tamil, a Dravidian language of South Asia, is a highly diglossic language with two very different registers in everyday use: Literary Tamil (preferred in writing and formal communication) and Spoken Tamil (confined to speech and informal media). Spoken Tamil is under-supported in modern NLP systems. In this paper, we release IruMozhi, a human-annotated dataset of parallel text in Literary and Spoken Tamil. We train classifiers on the task of identifying which variety a text belongs to. We use these models to gauge the availability of pretraining data in Spoken Tamil, to audit the composition of existing labelled datasets for Tamil, and to encourage future work on the variety.

Keywords

Cite

@article{arxiv.2311.07804,
  title  = {IruMozhi: Automatically classifying diglossia in Tamil},
  author = {Kabilan Prasanna and Aryaman Arora},
  journal= {arXiv preprint arXiv:2311.07804},
  year   = {2023}
}

Comments

4 pages main text, 7 total

R2 v1 2026-06-28T13:20:09.128Z