Related papers: Improved sampling and validation of frozen Gaussia…

Stochastic Gradient Descent (SGD) is one of the simplest and most popular stochastic optimization methods. While it has already been theoretically studied for decades, the classical analysis usually required non-trivial smoothness…

Stochastic gradient descent (SGD) algorithm is the method of choice in many machine learning tasks thanks to its scalability and efficiency in dealing with large-scale problems. In this paper, we focus on the shuffling version of SGD which…

Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential…

Stochastic variance reduced optimization methods are known to be globally convergent while they suffer from slow local convergence, especially when moderate or high accuracy is needed. To alleviate this problem, we propose an optimization…

We consider minimizing $f(x) = \mathbb{E}[f(x,\omega)]$ when $f(x,\omega)$ is possibly nonsmooth and either strongly convex or convex in $x$. (I) Strongly convex. When $f(x,\omega)$ is $\mu-$strongly convex in $x$, we propose a variable…

Reliable methods for obtaining time-dependent solutions of Langevin equations are in high demand in the field of non-equilibrium theory. In this paper, we present a new method based on variational superposed Gaussian approximation (VSGA)…

We study the Stochastic Gradient Descent (SGD) method in nonconvex optimization problems from the point of view of approximating diffusion processes. We prove rigorously that the diffusion process can approximate the SGD algorithm weakly…

Stochastic gradient descent (SGD) is perhaps the most prevalent optimization method in modern machine learning. Contrary to the empirical practice of sampling from the datasets without replacement and with (possible) reshuffling at each…

This paper formalizes and analyzes Gaussian smoothing applied to two prominent optimization methods: Stochastic Gradient Descent (GSmoothSGD) and Adam (GSmoothAdam) in deep learning. By attenuating small fluctuations, Gaussian smoothing…

This letter presents a non-parametric modeling approach for forecasting stochastic dynamical systems on low-dimensional manifolds. The key idea is to represent the discrete shift maps on a smooth basis which can be obtained by the diffusion…

For complex nonlinear systems, it is challenging to design algorithms that are fast, scalable, and give an accurate approximation of the stability region. This paper proposes a sampling-based approach to address these challenges. By…

An approach to the quantum-classical mechanics of phase space dependent operators, which has been proposed recently, is remodeled as a formalism for wave fields. Such wave fields obey a system of coupled non-linear equations that can be…

Few-shot relation extraction with none-of-the-above (FsRE with NOTA) aims at predicting labels in few-shot scenarios with unknown classes. FsRE with NOTA is more challenging than the conventional few-shot relation extraction task, since the…

A new algorithm is developed to tackle the issue of sampling non-Gaussian model parameter posterior probability distributions that arise from solutions to Bayesian inverse problems. The algorithm aims to mitigate some of the hurdles faced…

Here we outline and test an extension of the energy grained master equation (EGME) for treating nonadiabatic (NA) hopping between different potential energy surfaces, which enables us to model the competition between stepwise collisional…

In this paper we address the convergence of stochastic approximation when the functions to be minimized are not convex and nonsmooth. We show that the "mean-limit" approach to the convergence which leads, for smooth problems, to the ODE…

We propose a new metaheuristic training scheme that combines Stochastic Gradient Descent (SGD) and Discrete Optimization in an unconventional way. Our idea is to define a discrete neighborhood of the current SGD point containing a number of…

Using the Wang-Landau flat histogram Monte Carlo (FHMC) simulation technique, we were able to study two types of triangulated spherical surface models in which the two-dimensional extrinsic curvature energy is assumed in the Hamiltonian.…

Numerical simulations to evaluate thoracic aortic hemodynamics include a computational fluid dynamic (CFD) approach or fluid-structure interaction (FSI) approach. While CFD neglects the arterial deformation along the cardiac cycle by…

Multiscale dynamics are ubiquitous in applications of modern science. Because of time scale separation between relatively small set of slowly evolving variables and (typically) much larger set of rapidly changing variables, direct numerical…