Power-law Dynamic arising from machine learning
Machine Learning
2023-06-19 v1 Machine Learning
Abstract
We study a kind of new SDE that was arisen from the research on optimization in machine learning, we call it power-law dynamic because its stationary distribution cannot have sub-Gaussian tail and obeys power-law. We prove that the power-law dynamic is ergodic with unique stationary distribution, provided the learning rate is small enough. We investigate its first exist time. In particular, we compare the exit times of the (continuous) power-law dynamic and its discretization. The comparison can help guide machine learning algorithm.
Cite
@article{arxiv.2306.09624,
title = {Power-law Dynamic arising from machine learning},
author = {Wei Chen and Weitao Du and Zhi-Ming Ma and Qi Meng},
journal= {arXiv preprint arXiv:2306.09624},
year = {2023}
}
Comments
see https://doi.org/10.1007/978-981-19-4672-1