Related papers: Average Behaviour in Discrete-Time Imprecise Marko…
We study a general class of random walks driven by a uniquely ergodic Markovian environment. Under a coupling condition on the environment we obtain strong ergodicity properties and concentration inequalities for the environment as seen…
Recent large deviation results have provided general lower bounds for the fluctuations of time-integrated currents in the steady state of stochastic systems. A corollary are so-called thermodynamic uncertainty relations connecting precision…
At the nanoscale, random effects govern not only the dynamics of a physical system but may also affect its observation. This work introduces a novel paradigm for coarse graining that eschews the assignment of a unique coarse-grained…
Measurements of any property of a microscopic system are bound to show significant deviations from the average, due to thermal fluctuations. For time-integrated currents such as heat, work or entropy production in a steady state, it is in…
Ergodicity is a fundamental issue for a stochastic process. In this paper, we refine results on ergodicity for a general type of Markov chain to a specific type or the $GI/G/1$-type Markov chain, which has many interesting and important…
The classical Birkhoff ergodic theorem states that for an ergodic Markov process the limiting behaviour of the time average of a function (having finite $p$-th moment, $p\ge1$, with respect to the invariant measure) along the trajectories…
The central limit theorem for Markov chains generated by iterated function systems consisting of orientation preserving homeomorphisms of the interval is proved. We study also ergodicity of such systems.
We study the stability of a Markovian model of electricity production and consumption that incorporates production volatility due to renewables and uncertainty about actual demand versus planned production. We assume that the energy…
In this paper we consider the field of local times of a discrete-time Markov chain on a general state space, and obtain uniform (in time) upper bounds on the total variation distance between this field and the one of a sequence of $n$…
We study a finite time horizon Markov decision process (MDP) consisting of several groups of multi-action finite-state restless bandit processes, which are identical within each group. The bandit processes into different groups can be…
We find a general formula for the distribution of time averaged observables for weakly non-ergodic systems. Such type of ergodicity breaking is known to describe certain systems which exhibit anomalous fluctuations, e.g. blinking quantum…
We investigate the classical active pure exploration problem in Markov Decision Processes, where the agent sequentially selects actions and, from the resulting system trajectory, aims at identifying the best policy as fast as possible. We…
Statistical divergences are important tools in data analysis, information theory, and statistical physics, and there exist well known inequalities on their bounds. However, in many circumstances involving temporal evolution, one needs…
We consider ergodic backward stochastic differential equations in a discrete time setting, where noise is generated by a finite state Markov chain. We show existence and uniqueness of solutions, along with a comparison theorem. To obtain…
We exhibit an efficient procedure for testing, based on a single long state sequence, whether an unknown Markov chain is identical to or $\varepsilon$-far from a given reference chain. We obtain nearly matching (up to logarithmic factors)…
Suppose X and Y are two independent irreducible Markov chains on n states. We consider the intersection time, which is the first time their trajectories intersect. We show for reversible and lazy chains that the total variation mixing time…
We study a limit behavior of a sequence of Markov processes (or Markov chains) such that their distributions outside of any neighborhood of a "singular" point attract to some probability law. In any neighborhood of this point the behavior…
Let $X:=(X_t)_{t\geq 0}$ be an ergodic Markov process on $\real^d$, and $p>0$. We derive upper bounds of the $p$-Wasserstein distance between the invariant measure and the empirical measures of the Markov process $X$. For this we assume,…
We introduce the Lyapunov approach to optimal control problems of average risk-sensitive Markov control processes with general risk maps. Motivated by applications in particular to behavioral economics, we consider possibly non-convex risk…
In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition…