Related papers: Convergence Rate in Nonlinear Two-Time-Scale Stoch…
This paper is concerned with stability analysis of nonlinear time-varying systems by using Lyapunov function based approach. The classical Lyapunov stability theorems are generalized in the sense that the time-derivative of the Lyapunov…
We provide tight finite-time convergence bounds for gradient descent and stochastic gradient descent on quadratic functions, when the gradients are delayed and reflect iterates from $\tau$ rounds ago. First, we show that without stochastic…
We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i.e., deviation of the output of the algorithm from the equilibrium point of an…
We study weighted Tikhonov regularization for large-scale linear discrete ill-posed problems with random noise. Under a polynomial upper-bound assumption on the generalized eigenvalues of the discrete forward operator, we derive stochastic…
We propose a two-phase systematical framework for approximation algorithm design and analysis via Lyapunov function. The first phase consists of using Lyapunov function as an input and outputs a continuous-time approximation algorithm with…
We analyze the finite sample regret of a decreasing step size stochastic gradient algorithm. We assume correlated noise and use a perturbed Lyapunov function as a systematic approach for the analysis. Finally we analyze the escape time of…
We provide Lyapunov-like characterizations of boundedness and convergence of non-trivial solutions for a class of systems with unstable invariant sets. Examples of systems to which the results may apply include interconnections of stable…
Although stochastic approximation learning methods have been widely used in the machine learning literature for over 50 years, formal theoretical analyses of specific machine learning algorithms are less common because stochastic…
The theory of stochastic approximations form the theoretical foundation for studying convergence properties of many popular recursive learning algorithms in statistics, machine learning and statistical physics. Large deviations for…
We propose a mechanism which produces periodic variations of the degree of predictability in dynamical systems. It is shown that even in the absence of noise when the control parameter changes periodically in time, below and above the…
This paper considers the problem of finite-time stability for stochastic nonlinear systems. A new Lyapunov theorem of stochastic finite-time stability is proposed, and an important corollary is obtained. Some comparisons with the existing…
We investigate the statistical properties of Temporal Difference (TD) learning with Polyak-Ruppert averaging, arguably one of the most widely used algorithms in reinforcement learning, for the task of estimating the parameters of the…
We investigate an example of noise-induced stabilization in the plane that was also considered in (Gawedzki, Herzog, Wehr 2010) and (Birrell, Herzog, Wehr 2011). We show that despite the deterministic system not being globally stable, the…
We analyse the asymptotic properties of a continuous-time, two-timescale stochastic approximation algorithm designed for stochastic bilevel optimisation problems in continuous-time models. We obtain the weak convergence rate of this…
We study quasi-stationary distributions and quasi-limiting behavior of Markov chains in general reducible state spaces with absorption. We propose a set of assumptions dealing with particular situations where the state space can be…
We present an efficient and validated method for approximating the stationary measures of random dynamical systems with smooth additive noise. The approach leverages the strong regularizing properties of the associated transfer operator…
We prove a characterization of the support of the law of the solution for a stochastic wave equation with two-dimensional space variable, driven by a noise white in time and correlated in space. The result is a consequence of an…
This paper proposes a novel nonlinear sliding mode state feedback controller for perturbed second-order systems. In analogy to a linear proportional-derivative (PD) feedback control, the proposed nonlinear scheme uses the output of interest…
Stochastic approximation (SA) is an iterative algorithm for finding the fixed point of an operator using noisy samples and widely used in optimization and Reinforcement Learning (RL). The noise in RL exhibits a Markovian structure, and in…
Three similar convergence notions are considered. Two of them are the long established notions of convergent dynamics and incremental stability. The other is the more recent notion of contraction analysis. All three convergence notions…