Related papers: Potentials of skip-free Markov chains
Reversibility is a key property of Markov chains, central to algorithms such as Metropolis-Hastings and other MCMC methods. Yet many applications yield non-reversible chains, motivating the problem of approximating them by reversible ones…
We study perturbation theory and uniform ergodicity for discrete-time Markov chains on general state spaces in terms of the uniform moments of the first hitting times on some set. The methods we adopt are different from previous ones. For…
Continuous-time Markov chains on non-negative integers can be used for modeling biological systems, population dynamics, and queueing models. Qualitative behaviors of birth-and-death models, typical examples of such one-dimensional…
In this short paper, we consider discrete-time Markov chains on lattices as approximations to continuous-time diffusion processes. The approximations can be interpreted as finite difference schemes for the generator of the process. We…
The main contribution of this paper is to present a new sufficient condition for the subexponential asymptotics of the stationary distribution of a GI/GI/1-type Markov chain without jumps from level "infinity" to level zero. For simplicity,…
The aim of this article is to prove that diffusion processes in $\mathbb{R}^d$ with a drift can be approximated by suitable Markov chains on $n^{-1}\mathbb{Z}^d$. Moreover, we investigate sufficient conditions on the conductances which…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…
We study the limiting object of a sequence of Markov chains analogous to the limits of graphs, hypergraphs, and other objects which have been studied. Following a suggestion of Aldous, we assign to a sequence of finite Markov chains with…
The aim of this paper is twofold: - To give an elementary and self-contained proof of an explicit formula for the free energy for a general class of polymer chains interacting with an environment through periodic potentials. This…
Consideration is given to the three different analytical methods for the computation of upper bounds for the rate of convergence to the limiting regime of one specific class of (in)homogeneous continuous-time Markov chains. This class is…
A single queueing system with time-dependent exponentially distributed arrival processes and exponential machine processes (Kendall notation $M_t/M_t/1$) is analyzed. Modeling the time evolution for the discrete queue-length distribution by…
In this research paper, weighted / unweighted, directed / undirected graphs are associated with interesting Discrete Time Markov Chains (DTMCs) as well as Continuous Time Markov Chains (CTMCs). The equilibrium / transient behaviour of such…
Two approaches to studying the correlation functions of the binary Markov sequences are considered. The first of them is based on the study of probability of occurring different ''words'' in the sequence. The other one uses recurrence…
Markov chains are convenient means of generating realizations of networks with a given (joint or otherwise) degree distribution, since they simply require a procedure for rewiring edges. The major challenge is to find the right number of…
We propose a method based on continuous time Markov chain approximation to compute the distribution of Parisian stopping times and price Parisian options under general one-dimensional Markov processes. We prove the convergence of the method…
The Markov chain approximation of a one-dimensional symmetric diffusion is investigated in this paper. Given an irreducible reflecting diffusion on a closed interval with scale function $s$ and speed measure $m$, the approximating Markov…
It is well-known that discrete-time finite-state Markov Chains, which are described by one-sided conditional probabilities which describe a dependence on the past as only dependent on the present, can also be described as one-dimensional…
We introduce a class of Markov chains, that contains the model of stochastic approximation by averaging and non-averaging. Using martingale approximation method, we establish various deviation inequalities for separately Lipschitz functions…
The preparation of the stationary distribution of irreducible, time-reversible Markov chains is a fundamental building block in many heuristic approaches to algorithmically hard problems. It has been conjectured that quantum analogs of…