Related papers: Rates of memory loss for null recurrent Markov cha…
This paper establishes the first almost sure convergence rate and the first maximal concentration bound with exponential tails for general contractive stochastic approximation algorithms with Markovian noise. As a corollary, we also obtain…
In a Markov chain started at a state $x$, the hitting time $\tau(y)$ is the first time that the chain reaches another state $y$. We study the probability $\mathbf{P}_x(\tau(y) = t)$ that the first visit to $y$ occurs precisely at a given…
Consider a sequence $(\eta^N(t) :t\ge 0)$ of continuous-time, irreducible Markov chains evolving on a fixed finite set $E$, indexed by a parameter $N$. Denote by $R_N(\eta,\xi)$ the jump rates of the Markov chain $\eta^N_t$, and assume that…
Common to all different kinds of recurrent neural networks (RNNs) is the intention to model relations between data points through time. When there is no immediate relationship between subsequent data points (like when the data points are…
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous…
The problem of estimating an unknown discrete distribution from its samples is a fundamental tenet of statistical learning. Over the past decade, it attracted significant research effort and has been solved for a variety of divergence…
Consider a sequence of continuous-time irreducible reversible Markov chains and a sequence of initial distributions, $\mu_n$. The sequence is said to exhibit $\mu_n$-cutoff if the convergence to stationarity in total variation distance is…
We consider Markov chains on partially ordered sets that generalize the success-runs and remaining life chains in reliability theory. We find conditions for recurrence and transience and give simple expressions for the invariant…
Stationary ergodic processes with finite alphabets are estimated by finite memory processes from a sample, an n-length realization of the process, where the memory depth of the estimator process is also estimated from the sample using…
In this paper we develop a statistical estimation technique to recover the transition kernel $P$ of a Markov chain $X=(X_m)_{m \in \mathbb N}$ in presence of censored data. We consider the situation where only a sub-sequence of $X$ is…
In a finite state irreducible Markov chain with stationary probabilities \pi_i and mean first passage times m_(ij) (mean recurrence time when i = j) it was first shown by Kemeny and Snell (1960) that \sum_j \pi_j m_(ij) is a constant K, not…
We derive a generalization of the Perron-Frobenius theorem to time-varying row-stochastic matrices as follows: using Kolmogorov's concept of absolute probability sequences, which are time-varying analogs of principal eigenvectors, we…
We introduce the notion of order of magnitude reversibility (OM-reversibility) in Markov chains that are parametrized by a positive parameter $\ep$. OM-reversibility is a weaker condition than reversibility, and requires only the knowledge…
Generic non-Markovian quantum processes have infinitely long memory, implying an exact description that grows exponentially in complexity with observation time. Here, we present a finite memory ansatz that approximates (or recovers) the…
Catastrophe Markov chain population models have received a lot of attention in the recent past. We herewith consider two special cases of such models involving total disasters, both in discrete and in continuous-time. Depending on the…
Motivated by applications in telecommunications, computer scienceand physics, we consider a discrete-time Markov process withrestart. At each step the process eitherwith a positive probability restarts from a given distribution, orwith the…
Let $\Xi_n \subset \mathbb R^d$, $n\ge 1$, be a sequence of finite sets and consider a $\Xi_n$-valued, irreducible, reversible, continuous-time Markov chain $(X^{(n)}_t:t\ge 0)$. Denote by $\mathscr P(\mathbb R^d) $ the set of probability…
In this manuscript, new algebraic and analytic aspects of the orthogonal polynomials satisfying $R_{II}$ type recurrence relation given by \begin{align*} \mathcal{P}_{n+1}(x) = (x-c_n)\mathcal{P}_n(x)-\lambda_n…
The asymptotic variance is an important criterion to evaluate the performance of Markov chains, especially for the central limit theorems. We give the variational formulas for the asymptotic variance of discrete-time (non-reversible) Markov…
We suggest an approach to obtaining general two-sided bounds on the rate of convergence in terms of special "weighted" norms related to total variation. Some important classes of continuous-time Markov chains are considered:…