Hateful content detection is one of the areas where deep learning can and should make a significant difference. The Hateful Memes Challenge from Facebook helps fulfill such potential by challenging the contestants to detect hateful speech in multi-modal memes using deep learning algorithms. In this paper, we utilize multi-modal, pre-trained models VilBERT and Visual BERT. We improved models' performance by adding training datasets generated from data augmentation. Enlarging the training data set helped us get a more than 2% boost in terms of AUROC with the Visual BERT model. Our approach achieved 0.7439 AUROC along with an accuracy of 0.7037 on the challenge's test set, which revealed remarkable progress.
@article{arxiv.2105.13132,
title = {Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution},
author = {Yang Li and Zinc Zhang and Hutchin Huang},
journal= {arXiv preprint arXiv:2105.13132},
year = {2021}
}
Comments
Our code is available at: https://github.com/yangland/hatefulchallenge