Related papers: Metastable Markov chains
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential…
A new approach for optimal estimation of Markov chains with sparse transition matrices is presented.
We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation…
Markov chain Monte Carlo methods are a powerful tool for sampling equilibrium configurations in complex systems. One problem these methods often face is slow convergence over large energy barriers. In this work, we propose a novel method…
The analysis of many problems of interest associated with Markov chains, e.g. stationary distributions, moments of first passage time distributions and moments of occupation time random variables, involves the solution of a system of linear…
In this paper we describe the metastable behavior of solutions to a class of parabolic systems. In particular, we improve some results contained in \cite{MS} by using different techniques to describe the slow motion of the internal layers.…
In this paper, we establish novel concentration inequalities for additive functionals of geometrically ergodic Markov chains similar to Rosenthal inequalities for sums of independent random variables. We pay special attention to the…
This article surveys recent advancements of strategy designs for persistent robotic surveillance tasks with the focus on stochastic approaches. The problem describes how mobile robots stochastically patrol a graph in an efficient way where…
Continuous Time Markov Chain (CMTC) is widely used to describe and analyze systems in several knowledge areas. Steady state availability is one important analysis that can be made through Markov chain formalism that allows researchers…
One of the proposed solutions to the equilibrium selection problem for agents learning in repeated games is obtained via the notion of stochastic stability. Learning algorithms are perturbed so that the Markov chain underlying the learning…
Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The…
A definition of metastable states applicable to arbitrary finite state Markov processes satisfying detailed balance is discussed. In particular, we identify a crucial condition that distinguishes genuine metastable states from other types…
In 2013, Lu and Ren \cite {luren} considered anticipated backward stochastic differential equations driven by finite state, continuous time Markov chain noise and established the existence and uniqueness of the solutions of these equations…
For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…
We consider processes which are functions of finite-state Markov chains. It is well known that such processes are rarely Markov. However, such processes are often regular in the following sense: the distant past values of the process have…
A right continuous Markov chain is introduced in the noise terms of the three-dimensional stochastic Navier-Stokes equation, and we call such stochastic system as stochastic Navier-Stokes equation with Markov switching. In the present…
Given an infinitesimal perturbation of a discrete-time finite Markov chain, we seek the states that are stable despite the perturbation, \textit{i.e.} the states whose weights in the stationary distributions can be bounded away from $0$ as…
We study (backward) stochastic differential equations with noise coming from a finite state Markov chain. We show that, for the solutions of these equations to be `Markovian', in the sense that they are deterministic functions of the state…
In this paper we study the additive functionals of Markov chains via conditioning with respect to both past and future of the chain. We shall point out new sufficient projective conditions, which assure that the variance of partial sums of…