Related papers: Empirical processes of iterated maps that contract…
The paper is concerned with approximating the distribution of a sum W of n integer valued random variables Y_i, whose distributions depend on the state of an underlying Markov chain X. The approximation is in terms of a translated Poisson…
The analytic properties of the Markov operator associated to a random walk are common tools in the study of the behaviour and some probabilistic features related to the walk. In this paper we consider a class of Markov operators which…
In skew-product systems with contractive factors, all orbits asymptotically approach the graph of the so-called sync function; hence, the corresponding regularity properties primarily matter. In the literature, sync function Lipschitz…
We here elaborate on a quantitative argument to support the validity of the Collatz conjecture, also known as the (3x + 1) or Syracuse conjecture. The analysis is structured as follows. First, three distinct fixed points are found for the…
This paper studies a large number of homogeneous Markov decision processes where the transition probabilities and costs are coupled in the empirical distribution of states (also called mean-field). The state of each process is not known to…
Let $F$ be a class of functions on a probability space $(\Omega,\mu)$ and let $X_1,...,X_k$ be independent random variables distributed according to $\mu$. We establish high probability tail estimates of the form $\sup_{f \in F} |\{i :…
We consider reinforcement learning in parameterized Markov Decision Processes (MDPs), where the parameterization may induce correlation across transition probabilities or rewards. Consequently, observing a particular state transition might…
The embedding problem of Markov transition matrices into continuous-time Markov semigroups is a classic problem that regained a lot of impetus and activities in recent years. We consider it here for the following generalisation of the…
We provide a rather explicit formula for the resolvent of a~concatenation of $N$ processes in terms of their exit laws and certain probability measures characterizing the way the processes are concatenated. As an application, we prove an…
This paper considers the speed of convergence (mixing) of a finite Markov kernel $P$ with respect to the Kullback-Leibler divergence (entropy). Given a Markov kernel one defines either a discrete-time Markov chain (with the $n$-step…
We develop a general theory of Markov chains realizable as random walks on $\mathscr R$-trivial monoids. It provides explicit and simple formulas for the eigenvalues of the transition matrix, for multiplicities of the eigenvalues via…
We consider a general honest homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such…
The partially asymmetric exclusion process (PASEP) is an important model from statistical mechanics which describes a system of interacting particles hopping left and right on a one-dimensional lattice of N sites. It is partially asymmetric…
A finite ergodic Markov chain is said to exhibit cutoff if its distance to stationarity remains close to 1 over a certain number of iterations and then abruptly drops to near 0 on a much shorter time scale. Discovered in the context of card…
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…
We study the limiting behavior of interacting particle systems indexed by large sparse graphs, which evolve either according to a discrete time Markov chain or a diffusion, in which particles interact directly only with their nearest…
The successive discrete structures generated by a sequential algorithm from random input constitute a Markov chain that may exhibit long term dependence on its first few input values. Using examples from random graph theory and search…
We consider trap models on Z^d, namely continuous time Markov jump process on Z^d with embedded chain given by a generic discrete time random walk, and whose mean waiting time at x is given by tau_x, with tau = (tau_x, x in Z^d) a family of…
We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…
The mixing time of a Markov chain determines how fast the iterates of the Markov chain converge to the stationary distribution; however, it does not control the dependencies between samples along the Markov chain. In this paper, we study…