English

CALCS 2021 Shared Task: Machine Translation for Code-Switched Data

Computation and Language 2022-02-22 v1

Abstract

To date, efforts in the code-switching literature have focused for the most part on language identification, POS, NER, and syntactic parsing. In this paper, we address machine translation for code-switched social media data. We create a community shared task. We provide two modalities for participation: supervised and unsupervised. For the supervised setting, participants are challenged to translate English into Hindi-English (Eng-Hinglish) in a single direction. For the unsupervised setting, we provide the following language pairs: English and Spanish-English (Eng-Spanglish), and English and Modern Standard Arabic-Egyptian Arabic (Eng-MSAEA) in both directions. We share insights and challenges in curating the "into" code-switching language evaluation data. Further, we provide baselines for all language pairs in the shared task. The leaderboard for the shared task comprises 12 individual system submissions corresponding to 5 different teams. The best performance achieved is 12.67% BLEU score for English to Hinglish and 25.72% BLEU score for MSAEA to English.

Keywords

Cite

@article{arxiv.2202.09625,
  title  = {CALCS 2021 Shared Task: Machine Translation for Code-Switched Data},
  author = {Shuguang Chen and Gustavo Aguilar and Anirudh Srinivasan and Mona Diab and Thamar Solorio},
  journal= {arXiv preprint arXiv:2202.09625},
  year   = {2022}
}
R2 v1 2026-06-24T09:45:54.906Z