Contextual Text Embeddings for Twi
Abstract
Transformer-based language models have been changing the modern Natural Language Processing (NLP) landscape for high-resource languages such as English, Chinese, Russian, etc. However, this technology does not yet exist for any Ghanaian language. In this paper, we introduce the first of such models for Twi or Akan, the most widely spoken Ghanaian language. The specific contribution of this research work is the development of several pretrained transformer language models for the Akuapem and Asante dialects of Twi, paving the way for advances in application areas such as Named Entity Recognition (NER), Neural Machine Translation (NMT), Sentiment Analysis (SA) and Part-of-Speech (POS) tagging. Specifically, we introduce four different flavours of ABENA -- A BERT model Now in Akan that is fine-tuned on a set of Akan corpora, and BAKO - BERT with Akan Knowledge only, which is trained from scratch. We open-source the model through the Hugging Face model hub and demonstrate its use via a simple sentiment classification example.
Cite
@article{arxiv.2103.15963,
title = {Contextual Text Embeddings for Twi},
author = {Paul Azunre and Salomey Osei and Salomey Addo and Lawrence Asamoah Adu-Gyamfi and Stephen Moore and Bernard Adabankah and Bernard Opoku and Clara Asare-Nyarko and Samuel Nyarko and Cynthia Amoaba and Esther Dansoa Appiah and Felix Akwerh and Richard Nii Lante Lawson and Joel Budu and Emmanuel Debrah and Nana Boateng and Wisdom Ofori and Edwin Buabeng-Munkoh and Franklin Adjei and Isaac Kojo Essel Ampomah and Joseph Otoo and Reindorf Borkor and Standylove Birago Mensah and Lucien Mensah and Mark Amoako Marcel and Anokye Acheampong Amponsah and James Ben Hayfron-Acquah},
journal= {arXiv preprint arXiv:2103.15963},
year = {2021}
}
Comments
10 pages paper; Accepted at African NLP Workshop @ EACL 2021