Related papers: Hoeffding's lemma for Markov Chains and its applic…
Consider a Markov chain defined on a finite state space, X, that leaves invariant the uniform distribution on X, and whose transition probabilities are integer multiples of 1/Q, for some integer Q. I show how a simulation of n transitions…
The stability of iterations of affine linear maps $\Psi_{n}(x)=A_{n}x+B_{n}$, $n=1,2,\ldots$, is studied in the presence of a Markovian environment, more precisely, for the situation when $(A_{n},B_{n})_{n\ge 1}$ is modulated by an ergodic…
Gaussian processes are valuable tools for non-parametric modelling, where typically an assumption of stationarity is employed. While removing this assumption can improve prediction, fitting such models is challenging. In this work,…
We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…
We consider Markov-switching regression models, i.e. models for time series regression analyses where the functional relationship between covariates and response is subject to regime switching controlled by an unobservable Markov chain.…
The problem of sampling from the stationary distribution of a Markov chain finds widespread applications in a variety of fields. The time required for a Markov chain to converge to its stationary distribution is known as the classical…
This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986)…
We define a stochastic variant of the proximal point algorithm in the general setting of nonlinear (separable) Hadamard spaces for approximating zeros of the mean of a stochastically perturbed monotone vector field and prove its convergence…
Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential…
Consider the class of (functions of) strictly stationary Markov chains in which (i) the second moments are finite and (ii) absolute regularity (beta-mixing) is satisfied with exponential mixing rate. For (functions of) Markov chains in that…
We study the approximation of a (finite) continuous-time Markov chain by a Markov chain on a reduced state space, and we provide formal error bounds for the approximated transient distributions in the Wasserstein distance. These bounds…
We show that the Bernstein-Hoeffding method can be employed to a larger class of generalized moments. This class includes the exponential moments whose properties play a key role in the proof of a well-known inequality of Wassily Hoeffding,…
Let $(X_t)$ be a discrete time Markov chain on a general state space. It is well-known that if $(X_t)$ is aperiodic and satisfies a drift and minorization condition, then it converges to its stationary distribution $\pi$ at an exponential…
Functional inequalities such as the Poincar\'e and log-Sobolev inequalities quantify convergence to equilibrium in continuous-time Markov chains by linking generator properties to variance and entropy decay. However, many applications,…
Spectral methods have proven to be a highly effective tool in understanding the intrinsic geometry of a high-dimensional data set $\left\{x_i \right\}_{i=1}^{n} \subset \mathbb{R}^d$. The key ingredient is the construction of a Markov chain…
This paper investigates the supervised learning problem with observations drawn from certain general stationary stochastic processes. Here by \emph{general}, we mean that many stationary stochastic processes can be included. We show that…
We give recurrence and transience criteria for two cases of time-homogeneous Markov chains on the real line with transition kernel $p(x,dy)=f_x(y-x)dy$, where $f_x(y)$ are probability densities of symmetric distributions and, for large…
It has been well known for some time that for strictly stationary Markov chains that are ``reversible'', that special symmetry provides special extra features in the mathematical theory. This paper here is primarily a purely expository…
We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…
In a general setting we solve the following inverse problem: Given a positive operators $R$, acting on measurable functions on a fixed measure space $(X,\mathcal B_X)$, we construct an associated Markov chain. Specifically, starting with a…