Related papers: Obtaining Measure Concentration from Markov Contra…

We generalise the coarse Ricci curvature method of Ollivier by considering the coarse Ricci curvature of multiple steps in the Markov chain. This implies new spectral bounds and concentration inequalities. We also extend this approach to…

A nonnegative coarse Ricci curvature for a Markov chain and the existence of an attractive point implies the concentration of the invariant probability measure around this point. The mass outside balls centered at the attractive point, as a…

We provide explicit nonasymptotic estimates for the rate of convergence of empirical means of Markov chains, together with a Gaussian or exponential control on the deviations of empirical means. These estimates hold under a "positive…

During the last two decades, concentration inequalities have been the subject of exciting developments in various areas, including convex geometry, functional analysis, statistical physics, high-dimensional statistics, pure and applied…

The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…

Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The…

This paper establishes the first almost sure convergence rate and the first maximal concentration bound with exponential tails for general contractive stochastic approximation algorithms with Markovian noise. As a corollary, we also obtain…

The problem of nonlinear filtering has engendered a surprising number of mathematical techniques for its treatment. A notable example is the change-of--probability-measure method originally introduced by Kallianpur and Striebel to derive…

Metric coinduction is a form of coinduction that can be used to establish properties of objects constructed as a limit of finite approximations. One can prove a coinduction step showing that some property is preserved by one step of the…

We give a simple development of the concentration properties of compound Poisson measures on the nonnegative integers. A new modification of the Herbst argument is applied to an appropriate modified logarithmic-Sobolev inequality to derive…

Using terminologies of information geometry, we derive upper and lower bounds of the tail probability of the sample mean. Employing these bounds, we obtain upper and lower bounds of the minimum error probability of the 2nd kind of error…

We present a novel approach to quantizing Markov chains. The approach is based on the Markov chain coupling method, which is frequently used to prove fast mixing. Given a particular coupling, e.g., a grand coupling, we construct a…

We consider the problem of estimating the measure of subsets in very large networks. A prime tool for this purpose is the Markov Chain Monte Carlo (MCMC) algorithm. This algorithm, while extremely useful in many cases, still often suffers…

Measure-valued Markov chains have raised interest in Bayesian nonparametrics since the seminal paper by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579--585) where a Markov chain having the law of the Dirichlet process as unique…

This article shows how coupled Markov chains that meet exactly after a random number of iterations can be used to generate unbiased estimators of the solutions of the Poisson equation. Through this connection, we re-derive known unbiased…

The method of 'coupling from the past' permits exact sampling from the invariant distribution of a Markov chain on a finite state space. The coupling is successful whenever the stochastic dynamics are such that there is coalescence of all…

The concentration of measure prenomenon roughly states that, if a set $A$ in a product $\Omega^N$ of probability spaces has measure at least one half, ``most'' of the points of $\Omega^N$ are ``close'' to $A$. We proceed to a systematic…

We survey existing techniques to bound the mixing time of Markov chains. The mixing time is related to a geometric parameter called conductance which is a measure of edge-expansion. Bounds on conductance are typically obtained by a…

This work explores a novel perspective on solving nonconvex and nonsmooth optimization problems by leveraging sampling based methods. Instead of treating the objective function purely through traditional (often deterministic) optimization…

In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dataset, collected using a so-called logging policy. We develop…