Related papers: Average Behaviour in Discrete-Time Imprecise Marko…
We provide a criterion for establishing lower bounds on the rate of convergence in $f$-variation of a continuous-time ergodic Markov process to its invariant measure. The criterion consists of novel super- and submartingale conditions for…
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This is done by considering as basic uncertainty models the so-called credal sets that…
We derive general bounds on the probability that the empirical first-passage time $\overline{\tau}_n\equiv \sum_{i=1}^n\tau_i/n$ of a reversible ergodic Markov process inferred from a sample of $n$ independent realizations deviates from the…
Time estimation is a fundamental task that underpins precision measurement, global navigation systems, financial markets, and the organisation of everyday life. Many biological processes also depend on time estimation by nanoscale clocks,…
The stochastic processes underlying the growth and stability of biological and psychological systems reveal themselves when far from equilibrium. Far from equilibrium, nonergodicity reigns. Nonergodicity implies that the average outcome for…
The class of nonlinear Markov processes is characterized by the dependence of the current state of the process on its current distribution in addition to the dependence on the previous state. Due to this feature, these processes are…
We examine the non-Markovian nature of human mobility by exposing the inability of Markov models to capture criticality in human mobility. In particular, the assumed Markovian nature of mobility was used to establish a theoretical upper…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
Consider a stochastic nonlinear system controlled over a possibly noisy communication channel. An important problem is to characterize the largest class of channels for which there exist coding and control policies so that the closed-loop…
We consider the problem of maximizing the expected average reward obtained over an infinite time horizon by $n$ weakly coupled Markov decision processes. Our setup is a substantial generalization of the multi-armed restless bandit problem…
Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…
The goal of this paper is to develop a general method to establish conditional ergodicity of infinite-dimensional Markov chains. Given a Markov chain in a product space, we aim to understand the ergodic properties of its conditional…
Motivated by its connection to the limit behaviour of imprecise Markov chains, we introduce and study the so-called convergence of upper transition operators: the condition that for any function, the orbit resulting from iterated…
This paper contains two parts. In the first part, we study the ergodicity of periodic measures of random dynamical systems on a separable Banach space. We obtain that the periodic measure of the continuous time skew-product dynamical system…
In the framework of statistical mechanics the properties of macroscopic systems are deduced starting from the laws of their microscopic dynamics. One of the key assumptions in this procedure is the ergodic property, namely the equivalence…
Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…
This paper is concerned with ergodic properties of inhomogeneous Markov processes. Since the transition probabilities depend on initial times, the existing methods to obtain invariant measures for homogeneous Markov processes are not…
The use of higher-order stochastic processes such as nonlinear Markov chains or vertex-reinforced random walks is significantly growing in recent years as they are much better at modeling high dimensional data and nonlinear dynamics in…
In order to give quantitative estimates for approximating the ergodic limit, we investigate probabilistic limit behaviors of time-averaging estimators of numerical discretizations for a class of time-homogeneous Markov processes, by…
We study ergodic properties of some Markov chains models in random environments when the random Markov kernels that define the dynamic satisfy some usual drift and small set conditions but with random coefficients. In particular, we adapt a…