Related papers: Potentials of skip-free Markov chains
We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…
Consider longitudinal networks whose edges turn on and off according to a discrete-time Markov chain with exponential-family transition probabilities. We characterize when their joint distributions are also exponential families with the…
This paper considers an approximation usually used when implementing Ramaswami's recursion for the stationary distribution of the M/G/1-type Markov chain. The approximation is called the level-increment-truncation approximation because it…
Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…
An (upward) skip-free Markov chain with the set of nonnegative integers as state space is a chain for which upward jumps may be only of unit size; there is no restriction on downward jumps. In a 1987 paper, Brown and Shao determined, for an…
In high performance systems it is sometimes hard to build very large graphs that are efficient both with respect to memory and compute. This paper proposes a data structure called Markov-chain-priority-queue (MCPrioQ), which is a lock-free…
Markov Chains offer ideal conditions for the study and mathematical modelling of a certain kind of situations depending on random variables. The basic concepts of the corresponding theory were introduced by Markov in 1907 on coding literary…
"Quantum trajectories" are solutions of stochastic differential equations also called Belavkin or Stochastic Schr\"odinger Equations. They describe random phenomena in quantum measurement theory. Two types of such equations are usually…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
We develop Markov chain mixing time estimates for a class of Markov chains with restricted transitions. We assume transitions may occur along a cycle of $n$ nodes and on $n^\gamma$ additional edges, where $\gamma < 1$. We find that the…
Poisson's equation has a lot of applications in various areas. Usually it is hard to derive the explicit expression of the solution of Poisson's equation for a Markov chain on an infinitely many state space. We will present a computational…
We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…
The potential approach is a general and simple method for modelling interest rates, foreign exchange rates, and in principle other types of financial assets. This paper takes data on some liquid interest rate derivatives, and fits potential…
We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing…
This article is a lecture note on the potential theory of (possibly non-reversible) Markov processes and on the connection of this theory with quantitative analysis of the metastability of stochastic processes.
In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas…
We develop a general theory for Markov chains whose transition probabilities are the coefficients of descent operators on combinatorial Hopf algebras. These model the breaking-then-recombining of combinational objects. Examples include the…
A new approach is developed for evaluating the convergence rate for nonlinear Markov chains (MC) based on the recently developed spectral radius technique of markovian coupling for linear MC and the idea of small nonlinear perturbations of…
We consider the maximum entropy Markov chain inference approach to characterize the collective statistics of neuronal spike trains, focusing on the statistical properties of the inferred model. We review large deviations techniques useful…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…