Related papers: On Transformations of Markov Chains and Poisson Bo…

Changing time of simple continuous-time Markov counting processes by independent unit-rate Poisson processes results in Markov counting processes for which we provide closed-form transition rates via composition of trajectories and with…

We consider a discrete-time Markov chain $(X^t,Y^t)$, $t=0,1,2,...$, where the $X$-component forms a Markov chain itself. Assume that $(X^t)$ is Harris-ergodic and consider an auxiliary Markov chain ${\hat{Y}^t}$ whose transition…

We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then…

We prove a functional limit theorem for Markov chains that, in each step, move up or down by a possibly state dependent constant with probability $1/2$, respectively. The theorem entails that the law of every one-dimensional regular…

We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…

In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…

The spectral gap of a Markov chain can be bounded by the spectral gaps of constituent "restriction" chains and a "projection" chain, and the strength of such a bound is the content of various decomposition theorems. In this paper, we…

This paper is concerned with the solution of the optimal stopping problem associated to the valuation of Perpetual American options driven by continuous time Markov chains. We introduce a new dynamic approach for the numerical pricing of…

A stable-like Markov chain is a time-homogeneous Markov chain on the real line with the transition kernel $p(x,dy)=f_x(y-x)dy$, where the density functions $f_x(y)$, for large $|y|$, have a power-law decay with exponent $\alpha(x)+1$, where…

We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this…

We suggest a method for constructing positive harmonic functions for a wide class of transition kernels on $Z^+$. We also find natural conditions under which these functions have positive finite limits at infinity. Further, we apply our…

We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model…

We consider irreversible Markov chains on finite commutative rings randomly generated using both addition and multiplication. We restrict ourselves to the case where the addition is uniformly random and multiplication is arbitrary. We first…

In the investigation of limits of Markov chains, the presence of states which become instantaneous states in the limit may prevent the convergence of the chain in the Skorohod topology. We present in this article a weaker topology adapted…

Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The…

A block Markov chain is a Markov chain whose state space can be partitioned into a finite number of clusters such that the transition probabilities only depend on the clusters. Block Markov chains thus serve as a model for Markov chains…

In this article, we prove that a small random perturbation of dynamical system with multiple stable equilibria converges to a Markov chain whose states are neighborhoods of the deepest stable equilibria, under a suitable time-rescaling,…

For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…

We prove a large deviation principle on path space for a class of discrete time Markov processes whose state space is the intersection of a regular domain $\L\subset \R^d$ with some lattice of spacing $\e$. Transitions from $x$ to $y$ are…

We obtain non-Gaussian limit laws for one-dimensional random walk in a random environment assuming that the environment is a function of a stationary Markov process. This is an extension of the work of Kesten, M. Kozlov and Spitzer for…