Related papers: Time multidimensional Markov Renewal chains -- An …
This paper gathers together different conditions which are all equivalent to geometric ergodicity of time-homogeneous Markov chains on general state spaces. A total of 34 different conditions are presented (27 for general chains plus 7 just…
. Markov chains in time, such as simple random walks, are at the heart of probability. In space, due to the absence of an obvious definition of past and future, a range of definitions of Markovianity have been proposed. In this paper, after…
We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…
In this paper, we first extend the celebrated PageRank modification to a higher-order Markov chain. Although this system has attractive theoretical properties, it is computationally intractable for many interesting problems. We next study a…
Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…
The possibility of physics in multiple time dimensions is investigated. Drawing on recent work by Walter Craig and myself, I show that, contrary to conventional wisdom, there is a well-posed initial value problem--deterministic, stable…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…
Modeling the time evolution of discrete sets of items (e.g., genetic mutations) is a fundamental problem in many biomedical applications. We approach this problem through the lens of continuous-time Markov chains, and show that the…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
Markov Chains offer ideal conditions for the study and mathematical modelling of a certain kind of situations depending on random variables. The basic concepts of the corresponding theory were introduced by Markov in 1907 on coding literary…
The first motivation of this paper is to study stationarity and ergodic properties for a general class of time series models defined conditional on an exogenous covariates process. The dynamic of these models is given by an autoregressive…
Let $(S_n)_n$ be a $R^d$-valued random walk ($d\geq2$). Using Babillot's method [2], we give general conditions on the characteristic function of $S_n$ under which $(S_n)_n$ satisfies the same renewal theorem as the classical one obtained…
The recently established spectral Favard theorem for bounded banded matrices admitting a positive bidiagonal factorization is applied to a broader class of Markov chains with bounded banded transition matrices, extending beyond the…
We study time-changed Markov processes to speed up the convergence of Markov chain Monte Carlo (MCMC) algorithms. The time-changed process is defined by adjusting the speed of time of a base process via a user-chosen, state-dependent…
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
The evolutionary process has been modelled in many ways using both stochastic and deterministic models. We develop an algebraic model of evolution in a population of asexually reproducing organisms in which we represent a stochastic walk in…
In multi-state life insurance, an adequate balance between analytic tractability, computational efficiency, and statistical flexibility is of great importance. This might explain the popularity of Markov chain modelling, where matrix…
The creativity and emergence of biological and psychological behavior are nonlinear. However, that does not necessarily mean only that the measurements of the behaviors are curvilinear. Furthermore, the linear model might fail to reduce…
In his 2011 work, Maas has shown that the law of any time-reversible continuous-time Markov chain with finite state space evolves like a gradient flow of the relative entropy with respect to its stationary distribution. In this work we show…