Adapting to the Low-Resource Double-Bind: Investigating Low-Compute Methods on Low-Resource African Languages
Abstract
Many natural language processing (NLP) tasks make use of massively pre-trained language models, which are computationally expensive. However, access to high computational resources added to the issue of data scarcity of African languages constitutes a real barrier to research experiments on these languages. In this work, we explore the applicability of low-compute approaches such as language adapters in the context of this low-resource double-bind. We intend to answer the following question: do language adapters allow those who are doubly bound by data and compute to practically build useful models? Through fine-tuning experiments on African languages, we evaluate their effectiveness as cost-effective approaches to low-resource African NLP. Using solely free compute resources, our results show that language adapters achieve comparable performances to massive pre-trained language models which are heavy on computational resources. This opens the door to further experimentation and exploration on full-extent of language adapters capacities.
Keywords
Cite
@article{arxiv.2303.16985,
title = {Adapting to the Low-Resource Double-Bind: Investigating Low-Compute Methods on Low-Resource African Languages},
author = {Colin Leong and Herumb Shandilya and Bonaventure F. P. Dossou and Atnafu Lambebo Tonja and Joel Mathew and Abdul-Hakeem Omotayo and Oreen Yousuf and Zainab Akinjobi and Chris Chinenye Emezue and Shamsudeen Muhammad and Steven Kolawole and Younwoo Choi and Tosin Adewumi},
journal= {arXiv preprint arXiv:2303.16985},
year = {2023}
}
Comments
Accepted to AfricaNLP workshop at ICLR2023