Related papers: Hoeffding's Inequality for Markov Chains under Gen…
Despite substantial progress in non-equilibrium physics, steady-state (s.s.) probabilities remain intractable to analysis. For a Markov process, s.s. probabilities can be expressed in terms of transition rates using the Matrix-Tree theorem…
It is more important to estimate the rate of convergence to a stationary distribution rather than only to prove the existence one in many applied problems of reliability and queuing theory. This can be done via standard methods, but only…
We consider a bivariate Markov chain $Z=\{Z_k\}_{k \geq 1}=\{(X_k,Y_k)\}_{k \geq 1}$ taking values on product space ${\cal Z}={\cal X} \times{ \cal Y}$, where ${\cal X}$ is possibly uncountable space and ${\cal Y}=\{1,\ldots, |{\cal Y}|\}$…
In this paper, we investigate the concentration properties of cumulative reward in Markov Decision Processes (MDPs), focusing on both asymptotic and non-asymptotic settings. We introduce a unified approach to characterize reward…
We consider the problem of sampling a multimodal distribution with a Markov chain given a small number of samples from the stationary measure. Although mixing can be arbitrarily slow, we show that if the Markov chain has a $k$th order…
In the thesis we take the split chain approach to analyzing Markov chains and use it to establish fixed-width results for estimators obtained via Markov chain Monte Carlo procedures (MCMC). Theoretical results include necessary and…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
In this paper, we establish moment and Bernstein-type inequalities for additive functionals of geometrically ergodic Markov chains. These inequalities extend the corresponding inequalities for independent random variables. Our conditions…
In this short note we provide an elementary proof that a certain type of nonuniform sequential Doeblin minorization condition implies non-uniform sequential "geometric" ergodicity. Using this result several limit theorems for inhomogeneous…
We establish a simple variance inequality for U-statistics whose underlying sequence of random variables is an ergodic Markov Chain. The constants in this inequality are explicit and depend on computable bounds on the mixing rate of the…
Lifted probabilistic inference algorithms have been successfully applied to a large number of symmetric graphical models. Unfortunately, the majority of real-world graphical models is asymmetric. This is even the case for relational…
Motivated by the theory of inhomogeneous Markov chains, we determine a sufficient condition for the convergence to 0 of a general product formed from a sequence of real or complex matrices. When the matrices have a common invariant subspace…
We study the Out-of-Distribution (OOD) generalization in machine learning and propose a general framework that establishes information-theoretic generalization bounds. Our framework interpolates freely between Integral Probability Metric…
We investigate the complexity of covariance matrix estimation for Gibbs distributions based on dependent samples from a Markov chain. We show that when $\pi$ satisfies a Poincar\'e inequality and the chain possesses a spectral gap, we can…
The partial sum of the states of a Markov chain or more generally a Markov source is asymptotically normally distributed under suitable conditions. One of these conditions is that the variance is unbounded. A simple combinatorial…
We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An…
Sequential and quantum Monte Carlo methods, as well as genetic type search algorithms can be interpreted as a mean field and interacting particle approximations of Feynman-Kac models in distribution spaces. The performance of these…
With the growth of magnitude of multi-agent networks, distributed optimization holds considerable significance within complex systems. Convergence, a pivotal goal in this domain, is contingent upon the analysis of infinite products of…
We consider Markov chains on general state spaces in stationary random environment which are defined by a random mapping that is contractive up to a bounded perturbation. We prove their convergence to a limiting law, providing convergence…
We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…