Related papers: Concentration bounds for two time scale stochastic…
Stochastic approximation is a powerful class of algorithms with celebrated success. However, a large body of previous analysis focuses on stochastic approximations driven by contractive operators, which is not applicable in some important…
The nonlinear two-time-scale stochastic approximation is widely studied under conditions of bounded variances in noise. Motivated by recent advances that allow for variability linked to the current state or time, we consider state- and…
This paper develops a two-stage stochastic model to investigate evolution of random fields on the unit sphere $\bS^2$ in $\R^3$. The model is defined by a time-fractional stochastic diffusion equation on $\bS^2$ governed by a diffusion…
In this paper, we discretize the Caputo time derivative of order \alpha \in (0,1) using the Alikhanov scheme on a quasi-graded temporal mesh, and employ the Newton linearization method to approximate the nonlinear term. This yields a…
We present a method to overapproximate forward stochastic reach sets of discrete-time, stochastic nonlinear systems with interval geometry. This is made possible by extending the theory of mixed-monotone systems to incorporate stochastic…
Multi-time-scale stochastic approximation is an iterative algorithm for finding the fixed point of a set of $N$ coupled operators given their noisy samples. It has been observed that due to the coupling between the decision variables and…
A space discrete approximation to a highly nonlinear reaction-diffusion system endowed with a stochastic dynamical boundary condition is analyzed and the convergence of the discrete scheme to the solution to the corresponding continuum…
The paper is largely of a review nature. It considers two main methods used to study stability and obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable…
Previous studies on two-timescale stochastic approximation (SA) mainly focused on bounding mean-squared errors under diminishing stepsize schemes. In this work, we investigate {\it constant} stpesize schemes through the lens of Markov…
This letter investigates the convergence and concentration properties of the Stochastic Mirror Descent (SMD) algorithm utilizing biased stochastic subgradients. We establish the almost sure convergence of the algorithm's iterates under the…
This paper studies the moment boundedness of solutions of linear stochastic delay differential equations with distributed delay. For a linear stochastic delay differential equation, the first moment stability is known to be identical to…
We study systems on time scales that are generalizations of classical differential or difference equations. In this paper we consider linear systems and their small nonlinear perturbations. In terms of time scales and of eigenvalues of…
Two-sided bounds are explored for concentration functions and R\'enyi entropies in the class of discrete log-concave probability distributions. They are used to derive certain variants of the entropy power inequalities.
It is shown that under standard hypotheses, if stochastic approximation iterates remain tight, they converge with probability one to what their o.d.e. limit suggests. A simple test for tightness (and therefore a.s. convergence) is provided.…
This paper gives new concentration inequalities for the spectral norm of a wide class of matrix martingales in continuous time. These results extend previously established Freedman and Bernstein inequalities for series of random matrices to…
We use the two-time scale subordination in order to describe dynamical processes in continuous media with a long-term memory. Our consideration touches two physical examples in detail. First we study a temporal evolution of the species…
This work is concerned with a singularly perturbed stochastic nonlinear wave equation with a random dynamical boundary condition. A splitting skill is used to derive the approximating equation of the system in the sense of probability…
In this paper, we develop approximate dynamic programming methods for stochastic systems modeled as Markov Decision Processes, given both soft performance criteria and hard constraints in a class of probabilistic temporal logic called…
We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel…
Stochastic approximation is a class of algorithms that update a vector iteratively, incrementally, and stochastically, including, e.g., stochastic gradient descent and temporal difference learning. One fundamental challenge in analyzing a…