Related papers: A Resolvent Approach to Metastability
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…
This article shows how coupled Markov chains that meet exactly after a random number of iterations can be used to generate unbiased estimators of the solutions of the Poisson equation. Through this connection, we re-derive known unbiased…
For near-critical, transient Markov chains on the non-negative integers in the Lamperti regime, where the mean drift at $x$ decays as $1/x$ as $x \to \infty$, we quantify degree of transience via existence of moments for conditional return…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…
We study the problem of resilient strategies in the presence of uncertainty. Resilient strategies enable an agent to make decisions that are robust against disturbances. In particular, we are interested in those disturbances that are able…
We devise a Monte Carlo based method for detecting whether a non-negative Markov chain is stable for a given set of parameter values. More precisely, for a given subset of the parameter space, we develop an algorithm that is capable of…
A sequence of Markov chains is said to exhibit (total variation) cutoff if the convergence to stationarity in total variation distance is abrupt. We consider reversible lazy chains. We prove a necessary and sufficient condition for the…
A trajectorial large deviation principle is established in a mean field thermodynamic limit for a multiclass loss network with diminishing rates, which may have several stable equilibria. The large deviation limit is identified as a unique…
We consider a system of reaction-diffusion equations in a bounded interval of the real line, with emphasis on the metastable dynamics, whereby the time-dependent solution approaches its steady state in an asymptotically exponentially long…
Many large-scale software systems demonstrate metastable failures. In this class of failures, a stressor such as a temporary spike in workload causes the system performance to drop and, subsequently, the system performance continues to…
In this brief note, we investigate some constructions of Lyapunov functions for stochastic discrete-time stabilizable dynamical systems, in other words, controlled Markov chains. The main question here is whether a Lyapunov function in some…
We show that any loop-free Markov chain on a discrete space can be viewed as a determinantal point process. As an application, we prove central limit theorems for the number of particles in a window for renewal processes and Markov renewal…
We investigate integrability properties of processes with linear regressions and quadratic conditional variances. We establish the right order of dependence of which moments are finite on the parameter defined below, raising the question of…
Adaptive Monte Carlo methods can be viewed as implementations of Markov chains with infinite memory. We derive a general condition for the convergence of a Monte Carlo method whose history dependence is contained within the simulated…
We provide Lyapunov-like characterizations of boundedness and convergence of non-trivial solutions for a class of systems with unstable invariant sets. Examples of systems to which the results may apply include interconnections of stable…
Continuous Time Markov Chain (CMTC) is widely used to describe and analyze systems in several knowledge areas. Steady state availability is one important analysis that can be made through Markov chain formalism that allows researchers…
We propose some backward-forward martingale decompositions for functions of reversible Markov chains. These decompositions are used to prove the functional CLT for reversible Markov chains with asymptotically linear variance of partial…
This paper discusses the stabilizability, weak stabilizability, exact observability and robust quadratic stabilizability of linear stochastic control systems. By means of the spectrum technique of the generalized Lyapunov operator, a…
A classic reachability problem for safety of dynamic systems is to compute the set of initial states from which the state trajectory is guaranteed to stay inside a given constraint set over a given time horizon. In this paper, we leverage…
Markov chains and Markov decision processes (MDPs) are well-established probabilistic models. While finite Markov models are well-understood, analysing their infinite counterparts remains a significant challenge. Decisiveness has proven to…