Related papers: A Resolvent Approach to Metastability
The continuous evolution of a wide variety of systems, including continuous-time Markov chains and linear hybrid automata, can be described in terms of linear differential equations. In this paper we study the decision problem of whether…
We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…
Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…
We consider Markov chains that obey the following general non-linear state space model: $\Phi_{k+1} = F(\Phi_k, \alpha(\Phi_k, U_{k+1}))$ where the function $F$ is $C^1$ while $\alpha$ is typically discontinuous and $\{U_k: k \in…
We establish convergence to an invariant measure as time tends to infinity, for a large class of (possibly non-Markovian) stochastic volatility models. Our arguments are based on a novel coupling idea for Markov chains which also extends to…
An input-output approach to stability analysis is explored for networked systems with uncertain link dynamics. The main result consists of a collection of integral quadratic constraints, which together imply robust stability of the…
The stability of stochastic Model Predictive Control (MPC) subject to additive disturbances is often demonstrated in the literature by constructing Lyapunov-like inequalities that ensure closed-loop performance bounds and boundedness of the…
A new approach is developed for evaluating the convergence rate for nonlinear Markov chains (MC) based on the recently developed spectral radius technique of markovian coupling for linear MC and the idea of small nonlinear perturbations of…
We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…
This paper analyzes stochastic networks consisting of finite capacity nodes with different classes of requests which move according to some routing policy. The Markov processes describing these networks do not, in general, have…
Markov decision processes model systems subject to nondeterministic and probabilistic uncertainty. A plethora of verification techniques addresses variations of reachability properties, such as: Is there a scheduler resolving the…
A Markov chain approach is applied to determine the capacity of a general class of q-ary ICI-free constrained systems that satisfy an arbitrary count constraint.
We investigate the convergence to (quasi--)equilibrium of a density dependent Markov chain in~${\mathbb Z}^d$, whose drift satisfies a system of ordinary differential equations having an attractive fixed point. For a sequence of such…
A spectral method for identifying lumping in large Markov chains is presented. Identification of meta stable states is treated as a special case. The method is based on spectral analysis of a self-adjoint matrix that is a function of the…
We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which…
This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter…
We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained.…
We obtain complementary recurrence and transience criteria for processes $X=(X_n)_{n \ge 0}$ with values in $\mathbb R^d_+$ fulfilling a non-linear equation $X_{n+1}=MX_n+g(X_n)+ \xi_{n+1}$. Here $M$ denotes a primitive matrix having…
I show how any reversible Markov chain on a finite state space that is irreducible, and hence suitable for estimating expectations with respect to its invariant distribution, can be used to construct a non-reversible Markov chain on a…
We formalize and analyze the notions of stochastic monotonicity and realizable mono-tonicity for Markov Chains in continuous-time, taking values in a finite partially ordered set. Similarly to what happens in discrete-time, the two notions…