Related papers: Moderate deviation principle for ergodic Markov ch…

We prove a new inequality controlling the large deviations of the empirical measure of a Markov chain. This inequality is based on the martingale used by Donsker and Varadhan and the minimax theorem. It holds for convex sets and it requires…

A stable-like Markov chain is a time-homogeneous Markov chain on the real line with the transition kernel $p(x,dy)=f_x(y-x)dy$, where the density functions $f_x(y)$, for large $|y|$, have a power-law decay with exponent $\alpha(x)+1$, where…

Let $(X_n)_{n\ge 1}$ be a Markov chain on a measurable state space $X$, and let $S_n = \sum_{k=1}^n f(X_k)$ be the associated Markov walk. For $y>0$, denote by $\tau_y$ the first time at which $y+S_n$ becomes non-positive. Assuming that the…

Markov chains and Markov decision processes (MDPs) are well-established probabilistic models. While finite Markov models are well-understood, analysing their infinite counterparts remains a significant challenge. Decisiveness has proven to…

We consider a sequence $X^n=(X^n_t)_{t\ge 0},n\ge 1$ of semimartingales. Each $X^n$ is a weak solution to an It\^o equation with respect to a Wiener process and a Poissonian martingale measure and is in general non-Markovian process. For…

We study one-sided and $\alpha$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the…

Deterministic Markov Decision Processes (DMDPs) are a mathematical framework for decision-making where the outcomes and future possible actions are deterministically determined by the current action taken. DMDPs can be viewed as a finite…

A labelled Markov decision process (MDP) is a labelled Markov chain with nondeterminism; i.e., together with a strategy a labelled MDP induces a labelled Markov chain. The model is related to interval Markov chains. Motivated by…

We present a general framework for applying machine-learning algorithms to the verification of Markov decision processes (MDPs). The primary goal of these techniques is to improve performance by avoiding an exhaustive exploration of the…

Two popular classes of methods for approximate inference are Markov chain Monte Carlo (MCMC) and variational inference. MCMC tends to be accurate if run for a long enough time, while variational inference tends to give better approximations…

This paper establishes the quantitative stability of invariant measures $\mu_{\alpha}$ for $\mathbb{R}^d$-valued ergodic stochastic differential equations driven by rotationally invariant multiplicative $\alpha$-stable processes with…

Let $(X_{i}, \mathcal{F}_{i})_{i\geq 1}$ be a sequence of supermartingale differences and let $S_k=\sum_{i=1}^k X_i$. We give an exponential moment condition under which $P(\max_{1\leq k \leq n} S_k \geq n)=O(\exp\{-C_1 n^{\alpha}\}),$…

Markov chain Monte Carlo is widely used in a variety of scientific applications to generate approximate samples from intractable distributions. A thorough understanding of the convergence and mixing properties of these Markov chains can be…

This is the fourth in a series of articles devoted to showing that a typical covering map of large degree to a fixed, regular graph has its new adjacency eigenvalues within the bound conjectured by Alon for random regular graphs. In this…

A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…

Consider an nxn random matrix X with i.i.d. nonnegative entries with bounded density, mean m, and finite positive variance sigma^2. Let M be the nxn random Markov matrix with i.i.d. rows obtained from X by dividing each row of X by its sum.…

Markov decision process (MDP) is a decision making framework where a decision maker is interested in maximizing the expected discounted value of a stream of rewards received at future stages at various states which are visited according to…

We establish the weak large deviations principle for empirical measures of Markov chains on $\mathbb R^d$ under mild assumptions. In particular, no irreducibility is assumed and the initial measure may be arbitrary. The proof is entirely…

The decreasing Markov chain on \{1,2,3, \ldots\} with transition probabilities $p(j,j-i) \propto 1/i$ arises as a key component of the analysis of the beta-splitting random tree model. We give a direct and almost self-contained…

Large deviation theory is a branch of probability theory that is devoted to a study of the "rate" at which empirical estimates of various quantities converge to their true values. The object of study in this paper is the rate at which…