Accelerating Convergence in Global Non-Convex Optimization with Reversible Diffusion
Abstract
Langevin Dynamics has been extensively employed in global non-convex optimization due to the concentration of its stationary distribution around the global minimum of the potential function at low temperatures. In this paper, we propose to utilize a more comprehensive class of stochastic processes, known as reversible diffusion, and apply the Euler-Maruyama discretization for global non-convex optimization. We design the diffusion coefficient to be larger when distant from the optimum and smaller when near, thus enabling accelerated convergence while regulating discretization error, a strategy inspired by landscape modifications. Our proposed method can also be seen as a time change of Langevin Dynamics, and we prove convergence with respect to KL divergence, investigating the trade-off between convergence speed and discretization error. The efficacy of our proposed method is demonstrated through numerical experiments.
Cite
@article{arxiv.2305.11493,
title = {Accelerating Convergence in Global Non-Convex Optimization with Reversible Diffusion},
author = {Ryo Fujino},
journal= {arXiv preprint arXiv:2305.11493},
year = {2023}
}