Related papers: Convergence Rate in Nonlinear Two-Time-Scale Stoch…
We study the policy evaluation problem in multi-agent reinforcement learning, modeled by a Markov decision process. In this problem, the agents operate in a common environment under a fixed control policy, working together to discover the…
The influence of multiplicative white noise on the resonance capture of strongly nonlinear oscillatory systems under chirped-frequency excitations is investigated. It is assumed that the intensity of the perturbation decays polynomially…
We consider the nonparametric estimation problem of time-dependent multivariate functions observed in a presence of additive cylindrical Gaussian white noise of a small intensity. We derive minimax lower bounds for the $L^2$-risk in the…
This paper addresses the gradient flow -- the continuous-time representation of the gradient method -- with the smooth approximation of a non-differentiable objective function and presents convergence analysis framework. Similar to the…
Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes proximal gradient…
For complex nonlinear systems, it is challenging to design algorithms that are fast, scalable, and give an accurate approximation of the stability region. This paper proposes a sampling-based approach to address these challenges. By…
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…
This paper investigates the asymptotic behavior of stochastic recursive inclusions in the presence of non-zero, non-diminishing bias, a setting that frequently arises in zeroth-order optimization, stochastic approximation with…
In large-scale learning algorithms, the momentum term is usually included in the stochastic sub-gradient method to improve the learning speed because it can navigate ravines efficiently to reach a local minimum. However, step-size and…
This paper develops a new approach to the estimation of the degree of boundedness or stability of multidimensional nonlinear systems with time-dependent nonperiodic coefficients-an essential task in various engineering and natural science…
We claim that looking at probability distributions of \emph{finite time} largest Lyapunov exponents, and more precisely studying their large deviation properties, yields an extremely powerful technique to get quantitative estimates of…
We present a stability analysis framework for the general class of discrete-time linear switching systems for which the switching sequences belong to a regular language. They admit arbitrary switching systems as special cases. Using recent…
This article establishes the existence of Lyapunov functions for analyzing the stability of a class of state-constrained systems, and it describes algorithms for their numerical computation. The system model consists of a differential…
This paper investigates the weighted-averaging dynamic for unconstrained and constrained consensus problems. Through the use of a suitably defined adjoint dynamic, quadratic Lyapunov comparison functions are constructed to analyze the…
This paper is a comprehensive study of a long observed phenomenon of increase in the stability margin and so the rate of convergence of a class of linear systems due to time delay. We use Lambert W function to determine (a) in what systems…
This paper investigates the problem of consensus tracking control of discrete time multi-agent systems under binary-valued communication. Different from most existing studies on consensus tracking, the transmitted information between agents…
On the basis of a local-projective (LP) approach we develop a method of noise reduction in time series that makes use of nonlinear constraints appearing due to the deterministic character of the underlying dynamical system. The Delaunay…
We consider spatially extended conductance based neuronal models with noise described by a stochastic reaction diffusion equation with additive noise coupled to a control variable with multiplicative noise but no diffusion. We only assume a…
We study the popular distributed consensus method over networks composed of a number of densely connected clusters with a sparse connection between them. In these cluster networks, the method often constitutes two-time-scale dynamics, where…
The analysis in Part I revealed interesting properties for subgradient learning algorithms in the context of stochastic optimization when gradient noise is present. These algorithms are used when the risk functions are non-smooth and…