Related papers: Concentration bounds for two time scale stochastic…

We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…

We analyze the behavior of stochastic approximation algorithms where iterates, in expectation, progress towards an objective at each step. When progress is proportional to the step size of the algorithm, we prove exponential concentration…

In this paper we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of non-additive iterate-dependent Markov noise. It is shown that the recursion on each…

Two-timescale Stochastic Approximation (SA) algorithms are widely used in Reinforcement Learning (RL). Their iterates have two parts that are updated using distinct stepsizes. In this work, we develop a novel recipe for their finite sample…

We propose a two-point flux approximation finite-volume scheme for a stochastic non-linear parabolic equation with a multiplicative noise. The time discretization is implicit except for the stochastic noise term in order to be compatible…

We obtain optimal moment bounds for Birkhoff sums, and optimal concentration inequalities, for a large class of slowly mixing dynamical systems, including those that admit anomalous diffusion in the form of a stable law or a central limit…

This work revisits the constant stepsize stochastic approximation algorithm for tracking a slowly moving target and obtains a bound for the tracking error that is valid for the entire time axis, using the Alekseev non-linear variation of…

This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…

We consider a two-node queue modeled as a two-dimensional random walk. In particular, we consider the case that one or both queues have finite buffers. We develop an approximation scheme based on the Markov reward approach to error bounds…

We show a decomposition into the sum of a martingale and a deterministic quantity for time averages of the solutions to non-autonomous SDEs and for discrete-time Markov processes. In the SDE case the martingale has an explicit…

We consider the stochastic integrals of multivariate point processes and study their concentration phenomena. In particular, we obtain a Bernstein type of concentration inequality through Dol\'eans-Dade exponential formula and a uniform…

Through certain appropriate constructions, we establish periodic solutions in distribution for some stochastic differential equations with infinite-dimensional Levy noise. Additionally, we obtain the corresponding periodic measures and…

For stochastic approximation algorithms with discontinuous dynamics, it is shown that under suitable distributional assumptions, the interpolated iterates track a Fillipov solution of the limiting differential inclusion. In addition, we…

A scheme for stabilizing stochastic approximation iterates by adaptively scaling the step sizes is proposed and analyzed. This scheme leads to the same limiting differential equation as the original scheme and therefore has the same…

For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…

Discrete Markov random fields form a natural class of models to represent images and spatial data sets. The use of such models is, however, hampered by a computationally intractable normalising constant. This makes parameter estimation and…

Motivated by studies of indirect measurements in quantum mechanics, we investigate stochastic differential equations with a fixed point subject to an additional infinitesimal repulsive perturbation. We conjecture, and prove for an important…

We provide a brief tutorial on the use of concentration inequalities as they apply to system identification of state-space parameters of linear time invariant systems, with a focus on the fully observed setting. We draw upon tools from the…

We show sharpened forms of the concentration of measure phenomenon centered at first order stochastic expansions. The bound are based on second order difference operators and second order derivatives. Applications to functions on the…

In this paper, we establish maximal concentration bounds for the iterates generated by a stochastic approximation (SA) algorithm under a contractive operator with respect to some arbitrary norm (for example, the $\ell_\infty$-norm). We…