Related papers: Markov chains in a field of traps

This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The…

This paper focuses on the design of time-invariant memoryless control policies for fully observed controlled Markov chains, with a finite state space. Safety constraints are imposed through a pre-selected set of forbidden states. A state is…

The aim of this paper is to propose a methodology for testing general hypothesis in a Markovian setting with random sampling. A discrete Markov chain X is observed at random time intervals $\tau$ k, assumed to be iid with unknown…

In this paper we survey the almost sure central limit theorem and its functional form (quenched) for stationary and ergodic processes. For additive functionals of a stationary and ergodic Markov chain these theorems are known under the…

We determine the conditions for the existence of a pair of degenerate parity breaking separable eigenstates in general arrays of arbitrary spins connected through $XYZ$ couplings of arbitrary range and placed in a transverse field, not…

Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…

This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…

In this paper, we continue the investigation of quantum Markov states (QMS) and define their mean entropies. Such entropies are explicitly computed under certain conditions. The present work takes a huge leap forward at tackling one of the…

We calculate exact convergence times to reach random bipartite entanglement for various random protocols. The eigenproblem of a Markovian chain governing the process is mapped to a spin chain, thereby obtaining exact expression for the gap…

Consider the following Markov chain on permutations of length $n$. At each time step we choose a random position. If the letter at that position is smaller than the letter immediately to the left (cyclically) then these letters swap…

We derive explicit upper bounds for the $\bar{d}$-distance between a chain of infinite order and its canonical $k$-steps Markov approximation. Our proof is entirely constructive and involves a "coupling from the past" argument. The new…

We consider irreversible Markov chains on finite commutative rings randomly generated using both addition and multiplication. We restrict ourselves to the case where the addition is uniformly random and multiplication is arbitrary. We first…

We introduce a deterministic analogue of Markov chains that we call the hunger game. Like rotor-routing, the hunger game deterministically mimics the behavior of both recurrent Markov chains and absorbing Markov chains. In the case of…

Drawing on some recent results that provide the formalism necessary to definite stationarity for infinite random graphs, this paper initiates the study of statistical and learning questions pertaining to these objects. Specifically, a…

This paper considers the problem of finding strategies that satisfy a mixture of sure and threshold objectives in Markov decision processes. We focus on a single $\omega$-regular objective expressed as parity that must be surely met while…

We consider an infinite spin chain as a bipartite system consisting of the left and right half-chain and analyze entanglement properties of pure states with respect to this splitting. In this context we show that the amount of entanglement…

For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular,…

We suggest an approach to obtaining general two-sided bounds on the rate of convergence in terms of special "weighted" norms related to total variation. Some important classes of continuous-time Markov chains are considered:…

The partial sum of the states of a Markov chain or more generally a Markov source is asymptotically normally distributed under suitable conditions. One of these conditions is that the variance is unbounded. A simple combinatorial…

Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…