Related papers: Hoeffding's inequality for continuous-time Markov …
In this paper, we propose a novel stochastic process that serves as a natural discrete-time counterpart to the continuous-time model known as the ``Poisson hyperbolic staircase'' proposed by Levikson et al. (1999), and clarify its…
Sewer pipe systems are essential for social and economic welfare. Managing these systems requires robust predictive models for degradation behaviour. This study focuses on probability-based approaches, particularly Markov chains, for their…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…
We observe that the technique of Markov contraction can be used to establish measure concentration for a broad class of non-contracting chains. In particular, geometric ergodicity provides a simple and versatile framework. This leads to a…
Given noisy, partial observations of a time-homogeneous, finite-statespace Markov chain, conceptually simple, direct statistical inference is available, in theory, via its rate matrix, or infinitesimal generator, $\mathsf{Q}$, since $\exp…
In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition…
Quantum Markov chains (QMCs) are positive maps on a trace-class space describing open quantum dynamics on graphs. Such objects have a statistical resemblance with classical random walks, while at the same time it allows for internal…
The paper considers a stochastic differential equation of Duffing type with Markov coefficients. The existence of unpredictable solutions is considered. The unpredictability is a property of bounded functions characterized by unbounded…
Convergence rates of Markov chains have been widely studied in recent years. In particular, quantitative bounds on convergence rates have been studied in various forms by Meyn and Tweedie [Ann. Appl. Probab. 4 (1994) 981-1101], Rosenthal…
Let $(X_t)_{t = 0 }^{\infty}$ be an irreducible reversible discrete time Markov chain on a finite state space $\Omega $. Denote its transition matrix by $P$. To avoid periodicity issues (and thus ensuring convergence to equilibrium) one…
Computing smoothing distributions, the distributions of one or more states conditional on past, present, and future observations is a recurring problem when operating on general hidden Markov models. The aim of this paper is to provide a…
In a setting, where only "exit measures" are given, as they are associated with an arbitrary right continuous strong Markov process on a separable metric space, we provide simple criteria for the validity of Harnack inequalities for…
In this paper, we study moment and concentration inequalities for the spectral norm of sums of dependent random matrices. We establish novel Rosenthal-Burkholder inequalities for discrete-time matrix local martingales,…
Hoeffding-type exponential inequalities are obtained for the distribution tails of canonical von Mises' statistics of arbitrary order based on samples from a stationary sequence of random variables satisfying the {\varphi}-mixing condition.…
In this paper, we establish moment and Bernstein-type inequalities for additive functionals of geometrically ergodic Markov chains. These inequalities extend the corresponding inequalities for independent random variables. Our conditions…
A new approach to the steady state detection in the uniformization method of solving continuous time Markov chains is introduced. The method is particularly useful in solving inhomogenous CTMC's in multiple steps, where the desired error…
In this paper we examine the numerical approximation of the limiting invariant measure associated with Feynman-Kac formulae. These are expressed in a discrete time formulation and are associated with a Markov chain and a potential function.…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving…
Consider a Markov process $\{\Phi(t) : t\geq 0\}$ evolving on a Polish space ${\sf X}$. A version of the $f$-Norm Ergodic Theorem is obtained: Suppose that the process is $\psi$-irreducible and aperiodic. For a given function $f\colon{\sf…