Related papers: Sharp moderate maximal inequalities for upward ski…
The limiting probability distribution is one of the key characteristics of a Markov chain since it shows its long-term behavior. In this paper, for a higher order Markov chain, we establish some properties related to its exact limiting…
Interval Markov chains extend classical Markov chains with the possibility to describe transition probabilities using intervals, rather than exact values. While the standard formulation of interval Markov chains features closed intervals,…
In this paper we show that if large jumps of an It\^o-semimartingale $X$ have a finite $p$-moment, $p>0$, the radial part of its drift is dominated by $-|X|^\kappa$ for some $\kappa\geq -1$, and the balance condition $p+\kappa>1$ holds…
We establish existence of Markov chains of mean-field type with unbounded jump intensities by means of a fixed point argument using the Total Variation distance. We further show existence of nearly-optimal controls and, using a Markov chain…
We make a connection between the continuous time and lazy discrete time Markov chains through the comparison of cutoffs and mixing time in total variation distance. For illustration, we consider finite birth and death chains and provide a…
In this paper, we study the concentration properties of quadratic forms associated with Markov chains using the martingale decomposition method introduced by Atchad\'e and Cattaneo (2014). In particular, we derive concentration inequalities…
In Markov Decision Processes (MDPs) with intermittent state information, decision-making becomes challenging due to periods of missing observations. Linear programming (LP) methods can play a crucial role in solving MDPs, in particular,…
A possibly time-dependent transition intensity matrix or generator $(Q(t))$ characterizes the law of a Markov jump process (MP). For a time homogeneous MP, the transition probability matrix (TPM) can be expressed as a matrix exponential of…
We prove maximal inequalities for $L_q$-valued martingales obtained by stochastic integration with respect to compensated random measures. A version of these estimates for integrals with respect to compensated Poisson random measures were…
Markov chain Monte Carlo (MCMC) methods generate samples that are asymptotically distributed from a target distribution of interest as the number of iterations goes to infinity. Various theoretical results provide upper bounds on the…
Markov chains have long been used for generating random variates from spatial point processes. Broadly speaking, these chains fall into two categories: Metropolis-Hastings type chains running in discrete time and spatial birth-death chains…
We study the Markov chain Monte Carlo (MCMC) estimator for numerical integration for functions that do not need to be square integrable w.r.t. the invariant distribution. For chains with a spectral gap we show that the absolute mean error…
We consider the down/up crossing property of weighted Markov branching processes. The joint probability distribution of multi crossing numbers of such processes are obtained. In particular, for Markov branching processes, the probability…
In this paper we survey some recent results on the central limit theorem and its weak invariance principle for stationary sequences. We also describe several maximal inequalities that are the main tool for obtaining the invariance…
We consider Markov chains with random transition probabilities which, moreover, fluctuate randomly with time. We describe such a system by a product of stochastic matrices, $U(t)=M_t\cdots M_1$, with the factors $M_i$ drawn independently…
A novel approach is proposed to establish a sharp upper bound on the expected supremum of a separable martingale random field, serving as an alternative to classical universal chaining-based methods. The proposed approach begins by deriving…
Motivated by the study of the time evolution of random dynamical systems arising in a vast variety of domains --- ranging from physics to ecology ---, we establish conditions for the occurrence of a non-trivial asymptotic behaviour for…
We study the joint laws of a continuous, uniformly integrable martingale, its maximum, and its minimum. In particular, we give explicit martingale inequalities which provide upper and lower bounds on the joint exit probabilities of a…
Catastrophe Markov chain population models have received a lot of attention in the recent past. We herewith consider two special cases of such models involving total disasters, both in discrete and in continuous-time. Depending on the…
Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…