Related papers: Entropy Contractions in Markov Chains: Half-Step, …
We study the error introduced by entropy regularization in infinite-horizon discrete discounted Markov decision processes. We show that this error decreases exponentially in the inverse regularization strength, both in a weighted…
We consider a simple discrete-time Markov chain with values in $[0,\infty)^{Z^d}$. The Markov chain describes various interesting examples such as oriented percolation, directed polymers in random environment, time discretizations of binary…
We summarize some results of geometric measure theory concerning rectifiable sets and measures. Combined with the entropic chain rule for disintegrations (Vigneaux, 2021), they account for some properties of the entropy of rectifiable…
This PhD thesis deals with the Markov picture of developed turbulence from the theoretical point of view. The thesis consists of two parts. The first part introduces stochastic thermodynamics, the second part aims at transferring the…
We consider synchronizing properties of Markov decision processes (MDP), viewed as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in some…
In this paper, we develop an in-depth analysis of non-reversible Markov chains on denumerable state space from a similarity orbit perspective. In particular, we study the class of Markov chains whose transition kernel is in the similarity…
We study the Markov chain Monte Carlo (MCMC) estimator for numerical integration for functions that do not need to be square integrable w.r.t. the invariant distribution. For chains with a spectral gap we show that the absolute mean error…
Let $Q$ be a transition probability on a measurable space $E$, let $(X\_n)\_n$ be a Markov chain associated to $Q$, and let $\xi$ be a real-valued measurable function on $E$, and $S\_n = \sum\_{k=1}^{n} \xi(X\_k)$. Under functional…
This paper studies a class of mixed mean-field jump processes on an abstract state space $\Pi$, together with their associated $N$-particle systems. The dynamics consist of the superposition of an independent Markovian component and a…
Similarity-sensitive entropy measures the uncertainty of a probability law relative to a similarity kernel that encodes the distinguishability between states. We develop a measure-theoretic treatment covering both finite similarity matrices…
In this paper, we investigate a nonparametric approach to provide a recursive estimator of the transition density of a non-stationary piecewise-deterministic Markov process, from only one observation of the path within a long time. In this…
We show that the product or convex combination of two Markov operators with equivalent stationary measures need not have a stationary measure from the same measure class. More specifically, we exhibit examples of a hitherto undescribed…
In this article integro-differential Volterra equations whose convolution kernel depends on the vector variable are considered and a connection of these equations with a class of semi-Markov processes is established. The variable order…
For continuous-time Markov jump processes on irreducible networks with time-independent rate constants, we employ a transition-based formalism to express the long-time precision of a single integrated current over an observable channel in…
Markov process is widely applied in almost all aspects of literature, especially important for understanding non-equilibrium processes. We introduce a decomposition to general Markov process in this paper. This decomposition decomposes the…
This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand. We achieve the same by extending the lock-in probability…
The rate of entropy production by a stochastic process quantifies how far it is from thermodynamic equilibrium. Equivalently, entropy production captures the degree to which detailed balance and time-reversal symmetry are broken. Despite…
We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is…
Consider a sequence (indexed by n) of Markov chains Z^n in R^d characterized by transition kernels that approximately (in n) depend only on the rescaled state n^{-1} Z^n. Subject to a smoothness condition, such a family can be closely…
We introduce quantum versions of the $\chi^2$-divergence, provide a detailed analysis of their properties, and apply them in the investigation of mixing times of quantum Markov processes. An approach similar to the one presented in [1-3]…