Using Two Losses and Two Datasets Simultaneously to Improve TempoWiC Accuracy
Computation and Language
2022-12-16 v1 Artificial Intelligence
Abstract
WSD (Word Sense Disambiguation) is the task of identifying which sense of a word is meant in a sentence or other segment of text. Researchers have worked on this task (e.g. Pustejovsky, 2002) for years but it's still a challenging one even for SOTA (state-of-the-art) LMs (language models). The new dataset, TempoWiC introduced by Loureiro et al. (2022b) focuses on the fact that words change over time. Their best baseline achieves 70.33% macro-F1. In this work, we use two different losses simultaneously to train RoBERTa-based classification models. We also improve our model by using another similar dataset to generalize better. Our best configuration beats their best baseline by 4.23% and reaches 74.56% macroF1.
Keywords
Cite
@article{arxiv.2212.07669,
title = {Using Two Losses and Two Datasets Simultaneously to Improve TempoWiC Accuracy},
author = {Mohammad Javad Pirhadi and Motahhare Mirzaei and Sauleh Eetemadi},
journal= {arXiv preprint arXiv:2212.07669},
year = {2022}
}