Related papers: A Resolvent Approach to Metastability
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
We study a zero-range process with system-size dependent jump rates, which is known to exhibit a discontinuous condensation transition. Metastable homogeneous phases and condensed phases coexist in extended phase regions around the…
We propose a solution to a time-varying variant of Markov Decision Processes which can be used to address decision-theoretic planning problems for autonomous systems operating in unstructured outdoor environments. We explore the time…
This article is concerned with stability analysis and stabilization of randomly switched nonlinear systems. These systems may be regarded as piecewise deterministic stochastic systems: the discrete switches are triggered by a stochastic…
We discuss the efficient computation of performance, reliability, and availability measures for Markov chains; these metrics, and the ones obtained by combining them, are often called performability measures. We show that this computational…
We consider a symmetric finite-range contact process on $\mathbb{Z}$ with two types of particles (or infections), which propagate according to the same supercritical rate and die (or heal) at rate $1$. Particles of type 1 can occupy any…
This paper considers the problem of finding strategies that satisfy a mixture of sure and threshold objectives in Markov decision processes. We focus on a single $\omega$-regular objective expressed as parity that must be surely met while…
We study the condensation regime of the finite reversible inclusion process, i.e., the inclusion process on a finite graph $S$ with an underlying random walk that admits a reversible measure. We assume that the random walk kernel is…
We show the variational convergence of an irreversible Markov jump process describing a finite stochastic particle system to the solution of a countable infinite system of deterministic time-inhomogeneous quadratic differential equations…
In this article, we study the hierarchical structure of metastability in the reversible inclusion process. We fully characterize the third time scale of metastability subject to any underlying geometry of the system and prove that this is…
Inspired by semismooth Newton methods, we propose a general framework for designing solution methods with convergence guarantees for risk-averse Markov decision processes. Our approach accommodates a wide variety of risk measures by…
We obtain universal estimates on the convergence to equilibrium and the times of coupling for continuous time irreducible reversible finite-state Markov chains, both in the total variation and in the L^2 norms. The estimates in total…
This paper aims to develop the stability theory for singular stochastic Markov jump systems with state-dependent noise, including both continuous- and discrete-time cases. The sufficient conditions for the existence and uniqueness of a…
We show stability and locality of the minimal supersolution of a forward backward stochastic differential equation with respect to the underlying forward process under weak assumptions on the generator. The forward process appears both in…
We study stochastic monotonicity and propagation of order for Markov processes with respect to stochastic integral orders characterized by cones of functions satisfying $\Phi f \geq 0$ for some linear operator $\Phi$. We introduce a new…
Stability and safety are crucial in safety-critical control of dynamical systems. The reach-avoid-stay objectives for deterministic dynamical systems can be effectively handled by formal methods as well as Lyapunov methods with soundness…
We consider small perturbations of a dynamical system on the one-dimensional torus. We derive sharp estimates for the pre-factor of the stationary state, we examine the asymptotic behavior of the solutions of the Hamilton-Jacobi equation…
We prove the existence of solutions for the stochastic differential equation $dX_t=b(t,X_{t-})dZ_t+a(t,X_t)dt, X_0\in\R, t\ge 0,$ with only measurable coefficients $a$ and $b$ satisfying the condition $0<\mu\le |b(t,x)|\le \nu$ and…
Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite…
Semi-Markov processes generalize Markov processes by adding temporal memory effects as expressed by a semi-Markov kernel. We recall the path weight for a semi-Markov trajectory and the fact that thermodynamic consistency in equilibrium…