We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), including a ready-to-use software environment, as well as an experience report for data preparation and training. This paper introduces the first large-scale monolingual language models for Norwegian, based on both the ELMo and BERT frameworks. In addition to detailing the training process, we present contrastive benchmark results on a suite of NLP tasks for Norwegian. For additional background and access to the data, models, and software, please see http://norlm.nlpl.eu
@article{arxiv.2104.06546,
title = {Large-Scale Contextualised Language Modelling for Norwegian},
author = {Andrey Kutuzov and Jeremy Barnes and Erik Velldal and Lilja Øvrelid and Stephan Oepen},
journal= {arXiv preprint arXiv:2104.06546},
year = {2021}
}