English

MaxMatch-Dropout: Subword Regularization for WordPiece

Computation and Language 2022-09-12 v1

Abstract

We present a subword regularization method for WordPiece, which uses a maximum matching algorithm for tokenization. The proposed method, MaxMatch-Dropout, randomly drops words in a search using the maximum matching algorithm. It realizes finetuning with subword regularization for popular pretrained language models such as BERT-base. The experimental results demonstrate that MaxMatch-Dropout improves the performance of text classification and machine translation tasks as well as other subword regularization methods. Moreover, we provide a comparative analysis of subword regularization methods: subword regularization with SentencePiece (Unigram), BPE-Dropout, and MaxMatch-Dropout.

Keywords

Cite

@article{arxiv.2209.04126,
  title  = {MaxMatch-Dropout: Subword Regularization for WordPiece},
  author = {Tatsuya Hiraoka},
  journal= {arXiv preprint arXiv:2209.04126},
  year   = {2022}
}

Comments

Accepted to appear at COLING2022

R2 v1 2026-06-28T00:59:39.646Z