Related papers: Optimal Hoeffding bounds for discrete reversible M…
Consider a real diagonal deterministic matrix $X_n$ of size $n$ with spectral measure converging to a compactly supported probability measure. We perturb this matrix by adding a random finite rank matrix, with delocalized eigenvectors. We…
We analyse the $\ell^2(\pi)$-convergence rate of irreducible and aperiodic Markov chains with $N$-band transition probability matrix $P$ and with invariant distribution $\pi$. This analysis is heavily based on: first the study of the…
Let $\{\xi_1,\xi_2,\ldots\}$ be a sequence of independent but not necessarily identically distributed random variables. In this paper, the sufficient conditions are found under which the tail probability…
For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…
The paper considers a stochastic differential equation of Duffing type with Markov coefficients. The existence of unpredictable solutions is considered. The unpredictability is a property of bounded functions characterized by unbounded…
We obtain universal estimates on the convergence to equilibrium and the times of coupling for continuous time irreducible reversible finite-state Markov chains, both in the total variation and in the L^2 norms. The estimates in total…
In this article we derive the best possible upper bound for $E[\max{X_i}-\min_i{X_i}]$ under given means and variances on $n$ random variables $X_i$. The random vector $(X_1,...,X_n)$ is allowed to have any dependence structure, provided $E…
Reversibility is a key concept in Markov models and Master-equation models of molecular kinetics. The analysis and interpretation of the transition matrix encoding the kinetic properties of the model relies heavily on the reversibility…
We present a convex-concave reformulation of the reversible Markov chain estimation problem and outline an efficient numerical scheme for the solution of the resulting problem based on a primal-dual interior point method for monotone…
In this paper we establish a large deviations type estimate for strongly mixing Markov chains with respect to the Lp norm. As applications we derive such estimates for the iterates of a locally constant random cocycle with mixed rank, as…
Lyapunov exponents describe the asymptotic behavior of the singular values of large products of random matrices. A direct computation of these exponents is however often infeasible. By establishing a link between Lyapunov exponents and an…
We introduce a unified framework to estimate the convergence of Markov chains to equilibrium in Wasserstein distance. The framework can provide convergence bounds with rates ranging from polynomial to exponential, all derived from a…
In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dataset, collected using a so-called logging policy. We develop…
We consider a Wigner-type ensemble, i.e. large hermitian $N\times N$ random matrices $H=H^*$ with centered independent entries and with a general matrix of variances $S_{xy}=\mathbb E|H_{xy}|^2$. The norm of $H$ is asymptotically given by…
We prove new upper and lower bounds for sample complexity of finding an $\epsilon$-optimal policy of an infinite-horizon average-reward Markov decision process (MDP) given access to a generative model. When the mixing time of the…
The goal of this paper is to identify exponential convergence rates and to find computable bounds for them for Markov processes representing unreliable Jackson networks. First we use the bounds of Lawler and Sokal in order to show that, for…
We show bounds on total variation and $L^{\infty}$ mixing times, spectral gap and magnitudes of the complex valued eigenvalues of a general (non-reversible non-lazy) Markov chain with a minor expansion property. This leads to the first…
Affinity has proven to be a useful tool for quantifying the non-equilibrium character of time continuous Markov processes since it serves as a measure for the breaking of time reversal symmetry. It has recently been conjectured that the…
We study the problem of exponential mixing and large deviations for discrete-time Markov processes associated with a class of random dynamical systems. Under some dissipativity and regularisation hypotheses for the underlying deterministic…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…