Related papers: Contraction analysis of nonlinear random dynamical…
We consider a 2-dimensional stochastic differential equation in polar coordinates depending on several parameters. We show that if these parameters belong to a specific regime then the deterministic system explodes in finite time, but the…
It is by now established that, remarkably, the addition of noise to a nonlinear system may sometimes facilitate, rather than hamper the detection of weak signals. This phenomenon, usually referred to as stochastic resonance, was originally…
One calls attention to the fact that the stochastic physical systems are not random completely. They have both random and regular components of their evolution. Dynamic system is considered to be a special case of physical system with…
We consider a composite convex minimization problem associated with regularized empirical risk minimization, which often arises in machine learning. We propose two new stochastic gradient methods that are based on stochastic dual averaging…
Random metastability occurs when an externally forced or noisy system possesses more than one state of apparent equilibrium. This work investigates a class of random dynamical systems, arising from perturbing a one-dimensional piecewise…
This paper studies the design of neural network (NN)-based controllers for unknown nonlinear systems, using contraction analysis. A Neural Ordinary Differential Equation (NODE) system is constructed by approximating the unknown draft…
We study a conical extension of averaged nonexpansive operators and the role it plays in convergence analysis of fixed point algorithms. Various properties of conically averaged operators are systematically investigated, in particular, the…
We consider a distributionally robust formulation of stochastic optimization problems arising in statistical learning, where robustness is with respect to uncertainty in the underlying data distribution. Our formulation builds on…
This work focuses on the quantitative contraction rates for McKean-Vlasov stochastic differential equations (SDEs) with multiplicative noise. Under suitable conditions on the coefficients of the SDE, this paper derives explicit quantitative…
Investigating the dynamics of learning in machine learning algorithms is of paramount importance for understanding how and why an approach may be successful. The tools of physics and statistics provide a robust setting for such…
Across the scientific realm, we find ourselves subtracting or dividing stochastic signals. For instance, consider a stochastic realization, $x$, generated from the addition or multiplication of two stochastic signals $a$ and $b$, namely…
Finding the conditions that foster synchronization in networked oscillatory systems is critical to understanding a wide range of biological and mechanical systems. However, the conditions proved in the literature for synchronization in…
The objective of this paper is to derive the essential invariance and contraction properties for the geometric periodic systems, which can be formulated as a category of differential inclusions, and primarily rendered in the phase…
Iterative algorithms are ubiquitous in the field of data mining. Widely known examples of such algorithms are the least mean square algorithm, backpropagation algorithm of neural networks. Our contribution in this paper is an improvement…
We propose a combination of cluster analysis and stochastic process analysis to characterize high-dimensional complex dynamical systems by few dominating variables. As an example, stock market data are analyzed for which the dynamical…
The existence of the {\em typical set} is key for data compression strategies and for the emergence of robust statistical observables in macroscopic physical systems. Standard approaches derive its existence from a restricted set of…
This paper introduces a smoothed proximal Lagrangian method for minimizing a nonconvex smooth function over a convex domain with additional explicit convex nonlinear constraints. Two key features are 1) the proposed method is single-looped,…
Two algorithms are proposed, analyzed, and tested for solving continuous optimization problems with nonlinear equality constraints. Each is an extension of a stochastic momentum-based method from the unconstrained setting to the setting of…
We prove that Riemannian contraction in a supervised learning setting implies generalization. Specifically, we show that if an optimizer is contracting in some Riemannian metric with rate $\lambda > 0$, it is uniformly algorithmically…
In this paper we propose a wide class of truncated stochastic approximation procedures with moving random bounds. While we believe that the proposed class of procedures will find its way to a wider range of applications, the main motivation…