Related papers: Exponential Convergence Rates for Stochastically O…

New relations between ergodic rate, L_p convergence rates, and asymptotic behavior of tail probabilities for hitting times of a time homogeneous Markov process are established. For L_p convergence rates and related spectral and functional…

We consider the problem of computing the value and an optimal strategy for minimizing the expected termination time in one-counter Markov decision processes. Since the value may be irrational and an optimal strategy may be rather…

We study the problem of sequentially testing whether a given stochastic process is generated by a known Markov chain. Formally, given access to a stream of random variables, we want to quickly determine whether this sequence is a trajectory…

The goal of this paper is to identify exponential convergence rates and to find computable bounds for them for Markov processes representing unreliable Jackson networks. First we use the bounds of Lawler and Sokal in order to show that, for…

Let $(Z_n)$ be a supercritical branching process in a random environment $\xi$. We study the convergence rates of the martingale $W_n = Z_n/ E[Z_n| \xi]$ to its limit $W$. The following results about the convergence almost sur (a.s.), in…

Motivated by truncated EM method introduced by Mao (2015), a new explicit numerical method named modified truncated Euler-Maruyama method is developed in this paper. Strong convergence rates of the given numerical scheme to the exact…

Graded posets frequently arise throughout combinatorics, where it is natural to try to count the number of elements of a fixed rank. These counting problems are often $\#\textbf{P}$-complete, so we consider approximation algorithms for…

We consider a piecewise deterministic Markov decision process, where the expected exponential utility of total (nonnegative) cost is to be minimized. The cost rate, transition rate and post-jump distributions are under control. The state…

We give optimal convergence rates in the central limit theorem for a large class of martingale difference sequences with bounded third moments. The rates depend on the behaviour of the conditional variances and for stationary sequences the…

Let $Q$ be a transition probability on a measurable space $E$, let $(X\_n)\_n$ be a Markov chain associated to $Q$, and let $\xi$ be a real-valued measurable function on $E$, and $S\_n = \sum\_{k=1}^{n} \xi(X\_k)$. Under functional…

Consider the following communication scenario. An encoder observes a stochastic process and causally decides when and what to transmit about it, under a constraint on the expected number of bits transmitted per second. A decoder uses the…

We consider additive functionals of Markov processes in continuous time with general (metric) state spaces. We derive concentration bounds for their exponential moments and moments of finite order. Applications include diffusions,…

In this paper, we give quantitative bounds on the $f$-total variation distance from convergence of an Harris recurrent Markov chain on an arbitrary under drift and minorisation conditions implying ergodicity at a sub-geometric rate. These…

The study of time-inhomogeneous Markov jump processes is a traditional topic within probability theory that has recently attracted substantial attention in various applications. However, their flexibility also incurs a substantial…

In this paper, we propose a new policy iteration algorithm to compute the value function and the optimal controls of continuous time stochastic control problems. The algorithm relies on successive approximations using linear-quadratic…

Generalization of the Lorden's inequality is an excellent tool for obtaining strong upper bounds for the convergence rate for various complicated stochastic models. This paper demonstrates a method for obtaining such bounds for some…

In this paper, we establish the law of the iterated logarithm for a wide class of non-stationary, continuous-time Markov processes evolving on Polish spaces. Specifically, our result applies to certain additive functionals of processes…

Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…

Given the discrete-time sequence of nonnegative random variables, general dependencies between the exponential convergence of the expectations, exponential convergence of the trajectories and the logarithmic growth of the corresponding…

Many machine learning and optimization algorithms can be cast as instances of stochastic approximation (SA). The convergence rate of these algorithms is known to be slow, with the optimal mean squared error (MSE) of order $O(n^{-1})$. In…