Related papers: Exponential Convergence Rates for Stochastically O…
Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…
In this paper we obtain non-uniform exponential upper bounds for the rate of convergence of a version of the algorithm Context, when the underlying tree is not necessarily bounded. The algorithm Context is a well-known tool to estimate the…
For general, almost surely absorbed Markov processes, we obtain necessary and sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the…
We provide a general framework to study stochastic sequences related to individual learning in economics, learning automata in computer sciences, social learning in marketing, and other applications. More precisely, we study the asymptotic…
In this paper we consider the convergence of the conditional entropy to the entropy rate for Markov chains. Convergence of certain statistics of long range dependent processes, such as the sample mean, is slow. It has been shown in Carpio…
Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously…
Let $X$ be the constrained random walk on ${\mathbb Z}_+^2$ having increments $(1,0)$, $(-1,1)$, $(0,-1)$ with jump probabilities $\lambda(M_k)$, $\mu_1(M_k)$, and $\mu_2(M_k)$ where $M$ is an irreducible aperiodic finite state Markov…
We study the convergence rate to stationarity for a class of exchangeable partition-valued Markov chains called cut-and-paste chains. The law governing the transitions of a cut-and-paste chain are determined by products of i.i.d. stochastic…
We consider Markov processes in continuous time with state space $\posint^N$ and provide two sufficient conditions and one necessary condition for the existence of moments $E(\|X(t)\|^r)$ of all orders $r \in \nat$ for all $t \geq 0$. The…
The exponential rate of convergence and the Central Limit Theorem for some Markov operators are established. The operators correspond to iterated function systems which, for example, may be used to generalize the cell cycle model given by…
We study a Markov process with two components: the first component evolves according to one of finitely many underlying Markovian dynamics, with a choice of dynamics that changes at the jump times of the second component. The second…
Extreme value functionals of stochastic processes are inverse functionals of the first passage time -- a connection that renders their probability distribution functions equivalent. Here, we deepen this link and establish a framework for…
This paper establishes the first almost sure convergence rate and the first maximal concentration bound with exponential tails for general contractive stochastic approximation algorithms with Markovian noise. As a corollary, we also obtain…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…
We consider general Markov processes with absorption and provide criteria ensuring the exponential convergence in total variation of the distribution of the process conditioned not to be absorbed. The first one is based on two-sided…
We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point…
Let $(X,\p_x)$ be a continuous time Markov chain with finite or countable state space $S$ and let $T$ be its first passage time in a subset $D$ of $S$. It is well known that if $\mu$ is a quasi-stationary distribution relatively to $T$,…
The exponential upper bounds for the convergence rate of the distribution of restorable element with partially energized standby redundancy are founded, in the case when all working and repair times are bounded by exponential random…
We provide necessary and sufficient conditions for convergence of exponential integrals of Markov additive processes. Other than in the classical L\'evy case studied by Erickson and Maller we have to distinguish between almost sure…
In this paper it is shown that adaptive importance sampling algorithms converge at exponential rate for Markov chain expectation problems that admit a combination of a filtered estimator and a Markov zero-variance measure. It extends a…