相关论文: Stable states of perturbed Markov chains

We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…

We study singularly perturbed systems that exhibit input-to-state stability (ISS) with fixed-time properties in the presence of bounded disturbances. In these systems, solutions converge to the origin within a time frame independent of…

In this work the stability of perturbed linear time-varying systems is studied. The main features of the problem are threefold. Firstly, the time-varying dynamics is not required to be continuous but allowed to have jumps. Also the system…

This paper is concerned with stability analysis and synthesis for discrete-time linear systems with stochastic dynamics. Equivalence is first proved for three stability notions under some key assumptions on the randomness behind the…

Applications of stochastic models often involve the evaluation of steady-state performance, which requires solving a set of balance equations. In most cases of interest, the number of equations is infinite or even uncountable. As a result,…

We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…

A wide class of ``counting'' problems have been studied in Computer Science. Three typical examples are the estimation of - (i) the permanent of an $n\times n$ 0-1 matrix, (ii) the partition function of certain $n-$ particle Statistical…

Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…

The stability of iterations of affine linear maps $\Psi_{n}(x)=A_{n}x+B_{n}$, $n=1,2,\ldots$, is studied in the presence of a Markovian environment, more precisely, for the situation when $(A_{n},B_{n})_{n\ge 1}$ is modulated by an ergodic…

The stable operation of the electric power grid relies on a precisely synchronized state of all generators and machines. All machines rotate at exactly the same frequency with fixed phase differences, leading to steady power flows…

We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning…

Arguing about the equilibrium distribution of continuous-time Markov chains can be vital for showing properties about the underlying systems. For example in biological systems, bistability of a chemical reaction network can hint at its…

We propose a new definition of metastability of Markov processes on countable state spaces. We obtain sufficient conditions for a sequence of processes to be metastable. In the reversible case these conditions are expressed in terms of the…

For a class of linear switched systems in continuous time a controllability condition implies that state feedbacks allow to achieve almost sure stabilization with arbitrary exponential decay rates. This is based on the Multiplicative…

The current series of papers is concerned with stochastic stability of monotone dynamical systems by identifying the basic dynamical units that can survive in the presence of noise interference. In the first of the series, for the…

We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…

This paper is concerned with stability analysis of nonlinear time-varying systems by using Lyapunov function based approach. The classical Lyapunov stability theorems are generalized in the sense that the time-derivative of the Lyapunov…

Exponential stability of the nonlinear filtering equation is revisited, when the signal is a finite state Markov chain. An asymptotic upper bound for the filtering error due to incorrect initial condition is derived in the case of slowly…

We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…

We construct a marginally stable linear switching system in continuous time, in four dimensions and with three switching states, which is exponentially stable with respect to constant switching laws and which has a unique Barabanov norm,…