Fluctuation-dissipation Type Theorem in Stochastic Linear Learning
Machine Learning
2021-09-29 v1 Statistical Mechanics
Abstract
The fluctuation-dissipation theorem (FDT) is a simple yet powerful consequence of the first-order differential equation governing the dynamics of systems subject simultaneously to dissipative and stochastic forces. The linear learning dynamics, in which the input vector maps to the output vector by a linear matrix whose elements are the subject of learning, has a stochastic version closely mimicking the Langevin dynamics when a full-batch gradient descent scheme is replaced by that of stochastic gradient descent. We derive a generalized FDT for the stochastic linear learning dynamics and verify its validity among the well-known machine learning data sets such as MNIST, CIFAR-10 and EMNIST.
Cite
@article{arxiv.2106.02220,
title = {Fluctuation-dissipation Type Theorem in Stochastic Linear Learning},
author = {Manhyung Han and Jeonghyeok Park and Taewoong Lee and Jung Hoon Han},
journal= {arXiv preprint arXiv:2106.02220},
year = {2021}
}