Related papers: Finite-Time Error Bounds For Linear Stochastic App…

This paper considers time-average stochastic optimization, where a time average decision vector, an average of decision vectors chosen in every time step from a time-varying (possibly non-convex) set, minimizes a convex objective function…

The rate function for large deviations of the finite time Lyapunov exponent for the derived process in TM corresponding to a stochastic differential equation in M is related, via the Gartner-Ellis theorem, to the p-th moment Lyapunov…

With the objective of developing computational methods for stability analysis of switched systems, we consider the problem of finding the minimal lower bounds on average dwell-time that guarantee global asymptotic stability of the origin.…

This work presents the first finite-time analysis for the last-iterate convergence of average-reward $Q$-learning with an asynchronous implementation. A key feature of the algorithm we study is the use of adaptive stepsizes, which serve as…

We study the problem of least squares linear regression where the data-points are dependent and are sampled from a Markov chain. We establish sharp information theoretic minimax lower bounds for this problem in terms of…

In this work, we study finite-time stability of switched and hybrid systems in the presence of unstable modes. We present sufficient conditions in terms of multiple Lyapunov functions for the origin of the system to be finite time stable.…

Stiff and chaotic differential equations are challenging for time-stepping numerical methods. For explicit methods, the required time step resolution significantly exceeds the resolution associated with the smoothness of the exact solution…

This paper revisits the temporal difference (TD) learning algorithm for the policy evaluation tasks in reinforcement learning. Typically, the performance of TD(0) and TD($\lambda$) is very sensitive to the choice of stepsizes. Oftentimes,…

Stochastic gradient methods enable learning probabilistic models from large amounts of data. While large step-sizes (learning rates) have shown to be best for least-squares (e.g., Gaussian noise) once combined with parameter averaging,…

This paper presents four different ways of looking at the well-known Least Squares Temporal Differences (LSTD) algorithm for computing the value function of a Markov Reward Process, each of them leading to different insights: the…

We study stochastic delay differential equations (SDDE) where the coefficients depend on the moving averages of the state process. As a first contribution, we provide sufficient conditions under which a linear path functional of the…

This paper deals with the problem of finite-time learning for unknown discrete-time nonlinear systems' dynamics, without the requirement of the persistence of excitation. Two finite-time concurrent learning methods are presented to…

For the ordinary differential equation (ODE) $\dot{x}(t) = f(t,x)$, $x(0) = x_0$, $t\geq 0$, $x\in R^d$, assume $f$ to be at least continuous in $t$ and locally Lipshitz in $x$, and if necessary, several times continuously differentiable in…

This technical note studies Lyapunov-like conditions to ensure a class of dynamical systems to exhibit predefined-time stability. The origin of a dynamical system is predefined-time stable if it is fixed-time stable and an upper bound of…

Langevin simulation provides an effective way to study collisional effects in beams by reducing the six-dimensional Fokker-Planck equation to a group of stochastic ordinary differential equations. These resulting equations usually have…

This work proposes a general framework for capturing noise-driven transitions in spatially extended non-equilibrium systems and explains the emergence of coherent patterns beyond the instability onset. The framework relies on stochastic…

Policy evaluation in reinforcement learning is often conducted using two-timescale stochastic approximation, which results in various gradient temporal difference methods such as GTD(0), GTD2, and TDC. Here, we provide convergence rate…

We study the problem of system identification for stochastic continuous-time dynamics, based on a single finite-length state trajectory. We present a method for estimating the possibly unstable open-loop matrix by employing properly…

Stochastic Gradient Descent (SGD) is widely used in machine learning research. Previous convergence analyses of SGD under the vanishing step-size setting typically require Robbins-Monro conditions. However, in practice, a wider variety of…

In this paper we consider the problem of minimizing a quadratic functional for a discrete-time linear stochastic system with multiplicative noise, on a standard probability space, in infinite time horizon. We show that the necessary and…