Related papers: Moderate deviation principle for exponentially erg…

This paper considers large families of Markov chains (MCs) that are defined over a set of parameters with finite discrete domains. Such families occur in software product lines, planning under partial observability, and sketching of…

We study the ergodic theory of a one-parameter family of interval maps T_alpha arising from generalized continued fraction algorithms. First of all, we prove the dependence of the metric entropy of T_alpha to be Hoelder-continuous in the…

Consider longitudinal networks whose edges turn on and off according to a discrete-time Markov chain with exponential-family transition probabilities. We characterize when their joint distributions are also exponential families with the…

Consider a population of individuals belonging to an infinity number of types, and assume that type proportions follow the two-parameter Poisson-Dirichlet distribution. A sample of size n is selected from the population. The total number of…

We study a class of infinite-horizon average-cost Markov Decision Processes (MDPs) whose reward and transition structures are nearly separable. For the totally separable baseline (that is, with no perturbation), we derive an explicit…

We consider a Markov chain X_1, X_2, ..., X_n belonging to a class of iterated random functions, which is "one-step contracting" with respect to some distance d. If f is any separately Lipschitz function with respect to d, we use a well…

Robust Markov Decision Processes (MDPs) are a powerful framework for modeling sequential decision-making problems with model uncertainty. This paper proposes the first first-order framework for solving robust MDPs. Our algorithm interleaves…

Markov decision processes (MDPs) are known to be sensitive to parameter specification. Distributionally robust MDPs alleviate this issue by allowing for \emph{ambiguity sets} which give a set of possible distributions over parameter sets.…

We study the Markov chain Monte Carlo (MCMC) estimator for numerical integration for functions that do not need to be square integrable w.r.t. the invariant distribution. For chains with a spectral gap we show that the absolute mean error…

A formula for the transition density of a Markov process defined by an infinite-dimensional stochastic equation is given in terms of the Ornstein--Uhlenbeck bridge and a useful lower estimate on the density is provided. As a consequence,…

Let $\{X_n\}_{n\ge0}$ be a $V$-geometrically ergodic Markov chain. Given some real-valued functional $F$, define $M_n(\alpha):=n^{-1}\sum_{k=1}^nF(\alpha,X_{k-1},X_k)$, $\alpha\in\mathcal{A}\subset \mathbb {R}$. Consider an $M$ estimator…

We resolve the open question regarding the sample complexity of policy learning for maximizing the long-run average reward associated with a uniformly ergodic Markov decision process (MDP), assuming a generative model. In this context, the…

Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stopping times $\lambda_n$…

We view the classical Lindeberg principle in a Markov process setting to establish a probability approximation framework by the associated It\^{o}'s formula and Markov operator. As applications, we study the error bounds of the following…

Applying quantitative perturbation theory for linear operators, we prove non-asymptotic limit theorems for Markov chains whose transition kernel has a spectral gap in an arbitrary Banach algebra of functions X . The main results are…

Let $\Xi_n \subset \mathbb R^d$, $n\ge 1$, be a sequence of finite sets and consider a $\Xi_n$-valued, irreducible, reversible, continuous-time Markov chain $(X^{(n)}_t:t\ge 0)$. Denote by $\mathscr P(\mathbb R^d) $ the set of probability…

We prove explicit error bounds for Markov chain Monte Carlo (MCMC) methods to compute expectations of functions with unbounded stationary variance. We assume that there is a $p\in(1,2)$ so that the functions have finite $L_p$-norm. For…

We provide explicit nonasymptotic estimates for the rate of convergence of empirical means of Markov chains, together with a Gaussian or exponential control on the deviations of empirical means. These estimates hold under a "positive…

For a reversible and ergodic Markov chain $\{X_n,n\geq0\}$ with invariant distribution $\pi$, we show that a valid confidence interval for $\pi(h)$ can be constructed whenever the asymptotic variance $\sigma^2_P(h)$ is finite and positive.…

We prove a moderate deviations principles for the size of the largest connected component in a random $d$-uniform hypergraph. The key tool is a version of the exploration process, that is also used to investigate the giant component of an…