Related papers: Finite-Time Error Bounds For Linear Stochastic App…
Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving…
A numerical analysis for the fully discrete approximation of an operator Lyapunov equation related to linear SPDEs (stochastic partial differential equations) driven by multiplicative noise is considered. The discretization of the Lyapunov…
We study a stochastically perturbed version of the well-known Krasnoselski--Mann iteration for computing fixed points of nonexpansive maps in finite dimensional normed spaces. We discuss sufficient conditions on the stochastic noise and…
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…
Stochastic interpolants offer a robust framework for continuously transforming samples between arbitrary data distributions, holding significant promise for generative modeling. Despite their potential, rigorous finite-time convergence…
The focus of this paper is on stochastic variational inequalities (VI) under Markovian noise. A prominent application of our algorithmic developments is the stochastic policy evaluation problem in reinforcement learning. Prior…
We study equilibrium selection for invariant measures of stochastic dynamical systems with constant step size, under persistent noise and minimal moment assumptions, in a general quasi-Feller framework. Such dynamics arise in…
Two-time-scale stochastic approximation (SA) is an algorithm with coupled iterations which has found broad applications in reinforcement learning, optimization and game control. In this work, we derive mean squared error bounds for…
We derive ultimate precision bounds for estimating parameters encoded in \emph{time-dependent} Hamiltonians in the presence of general Markovian noise, allowing for arbitrary adaptive protocols with fast controls and noiseless ancillas.…
This paper proposes a methodology to estimate characteristic functions of stochastic differential equations that are defined over polynomials and driven by L\'evy noise. For such systems, the time evolution of the characteristic function is…
Previous studies on two-timescale stochastic approximation (SA) mainly focused on bounding mean-squared errors under diminishing stepsize schemes. In this work, we investigate {\it constant} stpesize schemes through the lens of Markov…
Finite time analysis of the continuous system is investigated through both stability and stabilization based on Sum of squares programming. A systematic approach is proposed to construct Lyapunov function and Control Lyapunov function for…
We consider off-policy temporal-difference (TD) learning methods for policy evaluation in Markov decision processes with finite spaces and discounted reward criteria, and we present a collection of convergence results for several…
This is a technical report that extends and clarifies the results presented in [1]. The model identification problem for asymptotically stable linear time invariant systems is considered. The system output is affected by an additive noise…
We consider optimization algorithms that successively minimize simple Taylor-like models of the objective function. Methods of Gauss-Newton type for minimizing the composition of a convex function and a smooth map are common examples. Our…
This paper provides a non-asymptotic analysis of linear stochastic approximation (LSA) algorithms with fixed stepsize. This family of methods arises in many machine learning tasks and is used to obtain approximate solutions of a linear…
Time estimation is a fundamental task that underpins precision measurement, global navigation systems, financial markets, and the organisation of everyday life. Many biological processes also depend on time estimation by nanoscale clocks,…
We develop a finite-dimensional sensitivity framework for studying stability in learning systems whose states include representations, parameters, and update variables. The central object is the \emph{Learning Stability Profile}, a…
This paper derives non-asymptotic error bounds for nonlinear stochastic approximation algorithms in the Wasserstein-$p$ distance. To obtain explicit finite-sample guarantees for the last iterate, we develop a coupling argument that compares…
This paper is concerned with stability analysis and synthesis for discrete-time linear systems with stochastic dynamics. Equivalence is first proved for three stability notions under some key assumptions on the randomness behind the…