相关论文: Transition records of stationary Markov chains

This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…

We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…

Consider an ergodic Markov chain on a countable state space for which the return times have exponential tails. We show that the stationary version of any such chain is a finitary factor of an i.i.d. process. A key step is to show that any…

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…

Thermalization is one of the most important phenomena in statistical physics. Often, the transition probabilities between different states in the phase space is or can be approximated by constants. In this case, the system can be described…

Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the…

This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…

Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…

Continuous-time Markov chains on non-negative integers can be used for modeling biological systems, population dynamics, and queueing models. Qualitative behaviors of birth-and-death models, typical examples of such one-dimensional…

We consider a strictly substochastic matrix or an stochastic matrix with absorbing states. By using quasi-stationary distributions one shows there is a canonical associated stationary Markov chain. Based upon $2-$stringing representation of…

We consider a stationary Markovian evolution with values on a disjointly partitioned set space $I\sqcup {\cal E}$. The evolution is visible (in the sense of knowing the transition probabilities) on the states in $I$ but not for the states…

Interpreting partial information collected from systems subject to noise is a key problem across scientific disciplines. Theoretical frameworks often focus on the dynamics of variables that result from coarse-graining the internal states of…

For a network of discrete states with a periodically driven Markovian dynamics, we develop an inference scheme for an external observer who has access to some transitions. Based on waiting-time distributions between these transitions, the…

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…

In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We…

We derive various exact results for Markovian systems that spontaneously relax to a non-equilibrium steady-state by using joint probability distributions symmetries of different entropy production decompositions. The analytical approach is…

We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…

The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…