Related papers: SDEs for Minimax Optimization
Majorization-minimization algorithms consist of iteratively minimizing a majorizing surrogate of an objective function. Because of its simplicity and its wide applicability, this principle has been very popular in statistics and in signal…
For finite-dimensional problems, stochastic approximation methods have long been used to solve stochastic optimization problems. Their application to infinite-dimensional problems is less understood, particularly for nonconvex objectives.…
Stochastic gradient methods have been a popular and powerful choice of optimization methods, aimed at minimizing functions. Their advantage lies in the fact that that one approximates the gradient as opposed to using the full Jacobian…
Stochastic optimization lies at the core of most statistical learning models. The recent great development of stochastic algorithmic tools focused significantly onto proximal gradient iterations, in order to find an efficient approach for…
Stochastic gradient descent (SGD) has been widely used in machine learning due to its computational efficiency and favorable generalization properties. Recently, it has been empirically demonstrated that the gradient noise in several deep…
We consider the problem of inferring latent stochastic differential equations (SDEs) with a time and memory cost that scales independently with the amount of data, the total length of the time series, and the stiffness of the approximate…
In this paper we present a new method for deriving It\^{o} stochastic delay differential equations (SDDEs) from delayed chemical master equations (DCMEs). Considering alternative formulations of SDDEs that can be derived from the same DCME,…
Stochastic differential equations have proved to be a valuable governing framework for many real-world systems which exhibit ``noise'' or randomness in their evolution. One quality of interest in such systems is the shape of their…
We discuss through multiple numerical examples the accuracy and efficiency of a micro-macro acceleration method for stiff stochastic differential equations (SDEs) with a time-scale separation between the fast microscopic dynamics and the…
Although the governing equations of many systems, when derived from first principles, may be viewed as known, it is often too expensive to numerically simulate all the interactions they describe. Therefore researchers often seek simpler…
Stochastic differential equations (SDEs) are popular tools to analyse time series data in many areas, such as mathematical finance, physics, and biology. They provide a mechanistic description of the phenomeon of interest, and their…
We study geometric stochastic differential equations (SDEs) and their approximations on Riemannian manifolds. In particular, we introduce a simple new construction of geometric SDEs, using which with bounded curvature. In particular, we…
We derive the stochastic version of the Magnus expansion for linear systems of stochastic differential equations (SDEs). The main novelty with respect to the related literature is that we consider SDEs in the It\^o sense, with progressively…
There have been recent efforts that combine seemingly disparate methods, extremum seeking (ES) optimization and partial differential equation (PDE) backstepping, to address the problem of model-free optimization with PDE actuator dynamics.…
We study the Stochastic Gradient Descent (SGD) method in nonconvex optimization problems from the point of view of approximating diffusion processes. We prove rigorously that the diffusion process can approximate the SGD algorithm weakly…
In this thesis, we extend the recently introduced theory of stochastic modified equations (SMEs) for stochastic gradient optimization algorithms. In Ch. 3 we study time-inhomogeneous SDEs driven by Brownian motion. For certain SDEs we prove…
In this paper, we study the optimal control of a discrete-time stochastic differential equation (SDE) of mean-field type, where the coefficients can depend on both a function of the law and the state of the process. We establish a new…
We derive novel algorithms for optimization problems constrained by partial differential equations describing multiscale particle dynamics, including non-local integral terms representing interactions between particles. In particular, we…
Most inverse problems from physical sciences are formulated as PDE-constrained optimization problems. This involves identifying unknown parameters in equations by optimizing the model to generate PDE solutions that closely match measured…
Stochastic Differential Equations (SDEs) are used as statistical models in many disciplines. However, intractable likelihood functions for SDEs make inference challenging, and we need to resort to simulation-based techniques to estimate and…