Related papers: Some universal estimates for reversible Markov cha…
We consider irreversible Markov chains on finite commutative rings randomly generated using both addition and multiplication. We restrict ourselves to the case where the addition is uniformly random and multiplication is arbitrary. We first…
We present a novel approach to quantizing Markov chains. The approach is based on the Markov chain coupling method, which is frequently used to prove fast mixing. Given a particular coupling, e.g., a grand coupling, we construct a…
The time reversal of a completely-positive, nonequilibrium discrete-time quantum Markov evolution is derived via a suitable adjointness relation. Space-time harmonic processes are introduced for the forward and reverse-time transition…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
We estimate the distance in total variation between the law of a finite state Markov process at time t, starting from a given initial measure, and its unique invariant measure. We derive upper bounds for the time to reach the equilibrium.…
Dealing with finite Markov chains in discrete time, the focus often lies on convergence behavior and one tries to make different copies of the chain meet as fast as possible and then stick together. There is, however, a very peculiar kind…
We propose an exact technique to calculate lower bounds of spectral gaps of discrete time reversible Markov chains on finite state sets. Spectral gaps are a common tool for evaluating convergence rates of Markov chains. As an illustration,…
The TCP window size process appears in the modeling of the famous Transmission Control Protocol used for data transmission over the Internet. This continuous time Markov process takes its values in [0, \infty), is ergodic and irreversible.…
In Monte-Carlo methods the Markov processes used to sample a given target distribution usually satisfy detailed balance, i.e. they are time-reversible. However, relatively recent results have demonstrated that appropriate reversible and…
This paper aims at improving the convergence to equilibrium of finite ergodic Markov chains via permutations and projections. First, we prove that a specific mixture of permuted Markov chains arises naturally as a projection under the KL…
A sequence of Markov chains is said to exhibit (total variation) cutoff if the convergence to stationarity in total variation distance is abrupt. We consider reversible lazy chains. We prove a necessary and sufficient condition for the…
If the state space of a homogeneous continuous-time Markov chain is too large, making inferences - here limited to determining marginal or limit expectations - becomes computationally infeasible. Fortunately, the state space of such a chain…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…
In this study, a new extension of the Markov Renewal theory is introduced by allowing time to evolve in multiple dimensions. The resulting chains are referred to as multi-time Markov Renewal chains and since this extension is new, the state…
We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation…
In this brief note, we find formulas for the distribution and the transition probability matrices of a stochastic process described as a time-reversion in a finite time window of a Markov chain, with cluster observation of the Markov state…
Markov chain Monte Carlo is widely used in a variety of scientific applications to generate approximate samples from intractable distributions. A thorough understanding of the convergence and mixing properties of these Markov chains can be…
There is a relation between the irreversibility of thermodynamic processes as expressed by the breaking of time-reversal symmetry, and the entropy production in such processes. We explain on an elementary mathematical level the relations…
The paper is largely of a review nature. It considers two main methods used to study stability and obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable…
This paper focuses on a class of continuous-time controlled Markov chains with time-inconsistent and distribution-dependent cost functional (in some appropriate sense). A new definition of time-inconsistent distribution-dependent…