Related papers: Imprecise Markov chains and their limit behaviour
We examine a constrained Markov decision process under uncertain transition probabilities, with the uncertainty modeled as deviations from observed transition probabilities. We construct the uncertainty set associated with the deviations…
This article expands the framework of Bayesian inference and provides direct probabilistic methods for approaching inference tasks that are typically handled with information theory. We treat Bayesian probability updating as a random…
The extremes of a univariate Markov chain with regulary varying stationary marginal distribution and asymptotically linear behavior are known to exhibit a multiplicative random walk structure called the tail chain. In this paper, we extend…
In this study, we address the central issue of statistical inference for Markov jump processes using discrete time observations. The primary problem at hand is to accurately estimate the infinitesimal generator of a Markov jump process, a…
Markov chains can be used to generate samples whose distribution approximates a given target distribution. The quality of the samples of such Markov chains can be measured by the discrepancy between the empirical distribution of the samples…
For a class of irreducible Markov chains with an infinitely countable set of states, we establish a new verifiable necessary and sufficient condition for recurrence and transience. We show that if one of the basic assumptions is not…
This work introduces a notion of approximate probabilistic trace equivalence for labelled Markov chains, and relates this new concept to the known notion of approximate probabilistic bisimulation. In particular this work shows that the…
Credal sets are sets of probability distributions that are considered as candidates for an imprecisely known ground-truth distribution. In machine learning, they have recently attracted attention as an appealing formalism for uncertainty…
The density-dependent Markov chain (DDMC) introduced in \cite{Kurtz1978} is a continuous time Markov process applied in fields such as epidemics, chemical reactions and so on. In this paper, we give moderate deviation principles of paths of…
Markov chains are a common framework for individual-based state and time discrete models in ecology and evolution. Their use, however, is largely limited to systems with a low number of states, since the transition matrices involved pose…
This paper deals with the problem of formulating an adaptive Model Predictive Control strategy for constrained uncertain systems. We consider a linear system, in presence of bounded time varying additive uncertainty. The uncertainty is…
We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pm\gamma$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases:…
Consider a finite irreducible Markov chain with invariant distribution $\pi$. We use the inner product induced by $\pi$ and the associated heat operator to simplify and generalize some results related to graph partitioning and the small-set…
We use the $f-divergence$ also called relative entropy as a measure of diversity between probability densities and review its basic properties. In the sequence we define a few objects which capture relevant information from the sample of a…
In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition…
There are known expressions to calculate the moments of the first passage time in Markov chains. Nevertheless, it is commonly forgotten that in most applications the parameters of the Markov chain are constructed using estimates based upon…
In this letter we announce rigorous results that elucidate the relation between metastable states and low-lying eigenvalues in Markov chains in a much more general setting and with considerable greater precision as was so far available.…
In this research the technology of complex Markov chains is applied to predict financial time series. The main distinction of complex or high-order Markov Chains and simple first-order ones is the existing of aftereffect or memory. The…
The distribution of the "mixing time" or the "time to stationarity" in a discrete time irreducible Markov chain, starting in state i, can be defined as the number of trials to reach a state sampled from the stationary distribution of the…
The cover time of a Markov chain on a finite state space is the expected time until all states are visited. We show that if the cover time of a discrete-time Markov chain with rational transitions probabilities is bounded, then it is a…