Consistent Batch Normalization for Weighted Loss in Imbalanced-Data Environment
Abstract
In this study, classification problems based on feedforward neural networks in a data-imbalanced environment are considered. Learning from an imbalanced dataset is one of the most important practical problems in the field of machine learning. A weighted loss function (WLF) based on a cost-sensitive approach is a well-known and effective method for imbalanced datasets. A combination of WLF and batch normalization (BN) is considered in this study. BN is considered as a powerful standard technique in the recent developments in deep learning. A simple combination of both methods leads to a size-inconsistency problem due to a mismatch between the interpretations of the effective size of the dataset in both methods. A simple modification to BN, called weighted BN (WBN), is proposed to correct the size mismatch. The idea of WBN is simple and natural. The proposed method in a data-imbalanced environment is validated using numerical experiments.
Cite
@article{arxiv.2001.01433,
title = {Consistent Batch Normalization for Weighted Loss in Imbalanced-Data Environment},
author = {Muneki Yasuda and Yeo Xian En and Seishirou Ueno},
journal= {arXiv preprint arXiv:2001.01433},
year = {2020}
}