Related papers: Approximation of heavy-tailed distributions via st…
We discover restrained numerical instabilities in current training practices of deep networks with stochastic gradient descent (SGD), and its variants. We show numerical error (on the order of the smallest floating point bit and thus the…
In stochastic systems, numerically sampling the relevant trajectories for the estimation of the large deviation statistics of time-extensive observables requires overcoming their exponential (in space and time) scarcity. The optimal way to…
We propose a time-space discretization scheme for quasi-linear parabolic PDEs. The algorithm relies on the theory of fully coupled forward--backward SDEs, which provides an efficient probabilistic representation of this type of equation.…
We present an explicit numerical approximation scheme, denoted by $\{X^n\}$, for the effective simulation of solutions $X$ to a multivariate stochastic differential equation (SDE) with a superlinearly growing $\kappa$-dissipative drift,…
Stochastic gradient descent (SGD) and its variants enable modern artificial intelligence. However, theoretical understanding lags far behind their empirical success. It is widely believed that SGD has a curious ability to avoid sharp local…
The area enclosed by the two-dimensional Brownian motion in the plane was studied by L\'evy, who found the characteristic function and probability density of this random variable. For other planar processes, in particular ergodic diffusions…
In 1966, Edward Nelson presented an interesting derivation of the Schrodinger equation using Brownian motion. Recently, this derivation is linked to the theory of optimal transport, which shows that the Schrodinger equation is a Hamiltonian…
We present an algorithm for the efficient sampling of conditional paths of stochastic differential equations (SDEs). While unconditional path sampling of SDEs is straightforward, albeit expensive for high dimensional systems of SDEs,…
In this article, we present a general methodology for stochastic control problems driven by the Brownian motion filtration including non-Markovian and non-semimartingale state processes controlled by mutually singular measures. The main…
Dynamical systems are essential to model various phenomena in physics, finance, economics, and are also of current interest in machine learning. A central modeling task is investigating parameter sensitivity, whether tuning atmospheric…
Robots rely on motion planning to navigate safely and efficiently while performing various tasks. In this paper, we investigate motion planning through Bayesian inference, where motion plans are inferred based on planning objectives and…
In this paper we propose a new methodology for decision-making under uncertainty using recent advancements in the areas of nonlinear stochastic optimal control theory, applied mathematics, and machine learning. Grounded on the fundamental…
We develop a new continuous-time stochastic gradient descent method for optimizing over the stationary distribution of stochastic differential equation (SDE) models. The algorithm continuously updates the SDE model's parameters using an…
Heavy-tailed distributions are found throughout many naturally occurring phenomena. We have reviewed the models of stochastic dynamics that lead to heavy-tailed distributions (and power law distributions, in particular) including the…
Considering generating samples with high rewards, we focus on optimizing deep neural networks parameterized stochastic differential equations (SDEs), the advanced generative models with high expressiveness, with policy gradient, the leading…
This paper introduces ergodic-risk criteria, which capture long-term cumulative risks associated with controlled Markov chains through probabilistic limit theorems--in contrast to existing methods that require assumptions of either finite…
In many physical or biological systems, diffusion can be described by Brownian motions with stochastic diffusion coefficients (DCs). In the present study, we investigate properties of the diffusion with a broad class of stochastic DCs with…
We study the convergence of a generic tamed Euler-Maruyama (EM) scheme for the kinetic type stochastic differential equations (SDEs) (also known as second order SDEs) with singular coefficients in both weak and strong probabilistic senses.…
Avikainen provided a sharp upper bound of the difference $\mathbb{E}[|g(X)-g(\widehat{X})|^{q}]$ by the moments of $|X-\widehat{X}|$ for any one-dimensional random variables $X$ with bounded density and $\widehat{X}$, and function of…
We generalize stochastic subgradient descent methods to situations in which we do not receive independent samples from the distribution over which we optimize, but instead receive samples that are coupled over time. We show that as long as…