Related papers: Metastability and small eigenvalues in Markov chai…
We establish a simple variance inequality for U-statistics whose underlying sequence of random variables is an ergodic Markov Chain. The constants in this inequality are explicit and depend on computable bounds on the mixing rate of the…
We consider a diffusion given by a small noise perturbation of a dynamical system driven by a potential function with a finite number of local minima. The classical results of Freidlin and Wentzell show that the time this diffusion spends…
Metastability is a phenomenon observed in stochastic systems which stay in a false-equilibrium within a region of its state space until the occurrence of a sequence of rare events that leads to an abrupt transition to a different region.…
The distribution of the "mixing time" or the "time to stationarity" in a discrete time irreducible Markov chain, starting in state i, can be defined as the number of trials to reach a state sampled from the stationary distribution of the…
In the last decades the problem of metastability has been attacked on rigorous grounds via many different approaches and techniques which are briefly reviewed in this paper. It is then useful to understand connections between different…
Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite…
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…
We investigate the close connection between metastability of the reversible diffusion process X defined by the stochastic differential equation dX_t=-\nabla F(X_t) dt+\sqrt2\epsilon dW_t,\qquad \epsilon >0, and the spectrum near zero of its…
The linear response of a dynamical system refers to changes to properties of the system when small external perturbations are applied. We consider the little-studied question of selecting an optimal perturbation so as to (i) maximise the…
We are interested in studying the sensitivity of diffusion processes or their approximations by Markov Chains with respect to a perturbation of the coefficients.
Statistical mechanics assumes that a quantum many-body system at low temperature can be effectively described by its Gibbs state. However, many complex quantum systems exist only as metastable states of dissipative open system dynamics,…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
This article provides a characterization of stability for switched nonlinear systems under average dwell-time constraints, in terms of necessary and sufficient conditions involving multiple Lyapunov functions. Earlier converse results focus…
In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We…
Let $(X_t)$ be a discrete time Markov chain on a general state space. It is well-known that if $(X_t)$ is aperiodic and satisfies a drift and minorization condition, then it converges to its stationary distribution $\pi$ at an exponential…
We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now…
In this paper we study the state-feedback stabilization of a discrete-time Markov jump linear system when the observation of the Markov chain of the system, called the Markov state, is time-randomized by another Markov chain. Embedding the…
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This is done by considering as basic uncertainty models the so-called credal sets that…
For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular,…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…