We demonstrate equivalence between the reinforcement learning problem and the supervised classification problem. We consequently equate the exploration exploitation trade-off in reinforcement learning to the dataset imbalance problem in supervised classification, and find similarities in how they are addressed. From our analysis of the aforementioned problems we derive a novel loss function for reinforcement learning and supervised classification. Scope Loss, our new loss function, adjusts gradients to prevent performance losses from over-exploitation and dataset imbalances, without the need for any tuning. We test Scope Loss against SOTA loss functions over a basket of benchmark reinforcement learning tasks and a skewed classification dataset, and show that Scope Loss outperforms other loss functions.
Cite
@article{arxiv.2308.04024,
title = {Scope Loss for Imbalanced Classification and RL Exploration},
author = {Hasham Burhani and Xiao Qi Shi and Jonathan Jaegerman and Daniel Balicki},
journal= {arXiv preprint arXiv:2308.04024},
year = {2023}
}
Comments
11 pages, 2 figures, under review for NeurIPS 2023