Related papers: Time inhomogeneous Markov chains with wave-like be…
We introduce a new partial order on the class of stochastically monotone Markov kernels having a given stationary distribution $\pi$ on a given finite partially ordered state space $\mathcal{X}$. When $K\preceq L$ in this partial order we…
It is well known that the distributions of hitting times in Markov chains are quite irregular, unless the limit as time tends to infinity is considered. We show that nevertheless for a typical finite irreducible Markov chain and for…
We introduce a new geometric approach that constructs a transition kernel of Markov chain. Our method always minimizes the average rejection rate and even reduce it to zero in many relevant cases, which cannot be achieved by conventional…
A 1-2 model configuration is a subset of edges of a hexagonal lattice satisfying the constraint that each vertex is incident to 1 or 2 edges. We introduce Markov chains to sample the 1-2 model configurations on 2D hexagonal lattice and…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
We obtain the memory kernel of the generalized Langevin equation, describing a particle interacting with longitudinal phonons in a liquid. The kernel is obtained analytically at T=0 Kelvin and numerically at T>0 Kelvin. We find that it…
General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic…
Path-dependence is a defining feature of many real-world systems, with applications ranging from population dynamics to rough volatility models and electricity spot prices. In stochastic Volterra equations (SVEs), such dependence is encoded…
Consider a Markov chain with finite state $\{0, 1, ..., d\}$. We give the generation functions (or Laplace transforms) of absorbing (passage) time in the following two situations : (1) the absorbing time of state $d$ when the chain starts…
Markov chain Monte Carlo methods have become popular in statistics as versatile techniques to sample from complicated probability distributions. In this work, we propose a method to parameterize and train transition kernels of Markov chains…
Let $(\xi_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and…
In this study, a new extension of the Markov Renewal theory is introduced by allowing time to evolve in multiple dimensions. The resulting chains are referred to as multi-time Markov Renewal chains and since this extension is new, the state…
We consider ergodic backward stochastic differential equations in a discrete time setting, where noise is generated by a finite state Markov chain. We show existence and uniqueness of solutions, along with a comparison theorem. To obtain…
We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…
We consider the first-passage percolation problem on the random graph with vertex set N\times{0,1}, edges joining vertices at Euclidean distance equal to unity and independent exponential edge weights. We provide a central limit theorem for…
In this paper we establish a diffusion limit for a multivariate continuous time Markov chain whose components are indexed by vertices of a finite graph. The components take values in a common finite set of non-negative integers and evolve…
We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…
Probability generating functions for first passage times of Markov chains are found using the method of collective marks. A system of equations is found which can be used to obtain moments of the first passage times.
A method of constructing Markov chains on finite state spaces is provided. The chain is specified by three constraints: stationarity, dependence and marginal distributions. The generalized Pythagorean theorem in information geometry plays a…
We discuss a wide class of time inhomogeneous quantum evolution which is represented by two-parameter family of completely positive trace-preserving maps. These dynamical maps are constructed as infinite series of jump processes. It is…