Related papers: Dynamics of continuous time Markov chains with app…
Motivated by applications in telecommunications, computer scienceand physics, we consider a discrete-time Markov process withrestart. At each step the process eitherwith a positive probability restarts from a given distribution, orwith the…
Arguing about the equilibrium distribution of continuous-time Markov chains can be vital for showing properties about the underlying systems. For example in biological systems, bistability of a chemical reaction network can hint at its…
This study introduces a novel approach for learning mixtures of Markov chains, a critical process applicable to various fields, including healthcare and the analysis of web users. Existing research has identified a clear divide in…
This paper considers continuous-time block-monotone Markov chains (BMMCs) and their block-augmented truncations. We first introduce the block monotonicity and block-wise dominance relation for continuous-time Markov chains, and then provide…
We suggest a method for constructing positive harmonic functions for a wide class of transition kernels on $Z^+$. We also find natural conditions under which these functions have positive finite limits at infinity. Further, we apply our…
Firing patterns in the central nervous system often exhibit strong temporal irregularity and heterogeneity in their time averaged response properties. Previous studies suggested that these properties are outcome of an intrinsic chaotic…
We develop a central limit theorem (CLT) for a non-parametric estimator of the transition matrices in controlled Markov chains (CMCs) with finite state-action spaces. Our results establish precise conditions on the logging policy under…
Based on the theory of stochastic chemical kinetics, the inherent randomness and stochasticity of biochemical reaction networks can be accurately described by discrete-state continuous-time Markov chains. The analysis of such processes is,…
We develop a general theory of Markov chains realizable as random walks on $\mathscr R$-trivial monoids. It provides explicit and simple formulas for the eigenvalues of the transition matrix, for multiplicities of the eigenvalues via…
In this paper we study Monte Carlo estimators based on the likelihood ratio approach for steady-state sensitivity. We first extend the result of Glynn and Olvera-Cravioto [doi:doi: 10.1287/stsy.2018.002] to the setting of continuous time…
We establish sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the existence and exponential ergodicity of the Q-process, the process…
In this paper we propose a model for open Markov chains that can be interpreted as a system of non-interacting particles evolving according to the rules of a Markov chain. The number of particles in the system is not constant, because we…
Stochastic chains represent a wide and key variety of phenomena in many branches of science within the context of Information Theory and Thermodynamics. They are typically approached by a sequence of independent events or by a memoryless…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of…
In this paper, we are interested in investigating the perturbation bounds for the stationary distributions for discrete-time or continuous-time Markov chains on a countable state space. For discrete-time Markov chains, two new norm-wise…
We introduce a class of continuous-time bivariate phase-type distributions for modeling dependencies from common shocks. The construction uses continuous-time Markov processes that evolve identically until an internal common-shock event,…
The asymptotic properties of some Markov processes associated to stochastic chemical reaction networks (CRNs) driven by the kinetics of the law of mass action are analyzed. The scaling regime introduced in the paper assumes that the norm of…
Within systems biology there is an increasing interest in the stochastic behavior of genetic and biochemical reaction networks. An appropriate stochastic description is provided by the chemical master equation, which represents a continuous…
We consider a class of continuous time Markov chains on $\Z^d$. These chains are the discrete space analogue of Markov processes with jumps. Under some conditions, we show that harmonic functions associated with these Markov chains are…