High-Dimensional Non-Convex Landscapes and Gradient Descent Dynamics
Disordered Systems and Neural Networks
2023-11-13 v2 Statistical Mechanics
Abstract
In these lecture notes we present different methods and concepts developed in statistical physics to analyze gradient descent dynamics in high-dimensional non-convex landscapes. Our aim is to show how approaches developed in physics, mainly statistical physics of disordered systems, can be used to tackle open questions on high-dimensional dynamics in Machine Learning.
Cite
@article{arxiv.2308.03754,
title = {High-Dimensional Non-Convex Landscapes and Gradient Descent Dynamics},
author = {Tony Bonnaire and Davide Ghio and Kamesh Krishnamurthy and Francesca Mignacco and Atsushi Yamamura and Giulio Biroli},
journal= {arXiv preprint arXiv:2308.03754},
year = {2023}
}
Comments
Lectures given by G. Biroli at the 2022 Les Houches Summer School "Statistical Physics and Machine Learning"