Related papers: Limit Behaviour of Upper and Lower Expected Time A…
Predefined-time stability enables convergence within a user-specified time independent of initial conditions. Existing results are predominantly based on autonomous Lyapunov inequalities, where the predefined-time is realized through…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…
In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition…
We study a new technique for the asymptotic analysis of heavy-tailed systems conditioned on large deviations events. We illustrate our approach in the context of ruin events of multidimensional regularly varying random walks. Our approach…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a…
This paper presents a synthesis approach aiming to guarantee a minimum upper-bound for the time taken to reach a target set of non-zero measure that encompasses the origin, while taking into account uncertainties and input and state…
The convergence rate of a Markov chain to its stationary distribution is typically assessed using the concept of total variation mixing time. However, this worst-case measure often yields pessimistic estimates and is challenging to infer…
We introduce a unified operator-theoretic framework for analyzing mixing times of finite-state ergodic Markov chains that applies to both reversible and non-reversible dynamics. The central object in our analysis is the projected transition…
In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to…
Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving…
Oftentimes in practice, the observed process changes statistical properties at an unknown point in time and the duration of a change is substantially finite, in which case one says that the change is intermittent or transient. We provide an…
The paper presents two results. The first one provides separate conditions for the upper and lower estimate of the distribution of the exit time from balls of a random walk on a weighted graph. The main result of the paper is that the lower…
Motivated by applications to COVID dynamics, we describe a branching process in random environments model $\{Z_n\}$ whose characteristics change when crossing upper and lower thresholds. This introduces a cyclical path behavior involving…
We show that under local detailed balance the expected entropy production rate is always bounded in terms of the dynamical activity. The activity refers to the time-symmetric contribution in the action functional for path-space…
This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a…
We consider triangular arrays of Markov chains that converge weakly to a diffusion process. Local limit theorems for transition densities are proved. The observation time [0,T] may be fixed or lim n T = 0, where nh = T and h is a mesh…
Analyzing Event-Triggered Control's (ETC) sampling behaviour is of paramount importance, as it enables formal assessment of its sampling performance and prediction of its sampling patterns. In this work, we formally analyze the sampling…
Understanding how local perturbations induce the transient dynamics of a network of coupled units is essential to control and operate such systems. Often a perturbation initiated in one unit spreads to other units whose dynamical state they…
Consider a Markov chain $\{X_n\}_{n\ge 0}$ with an ergodic probability measure $\pi$. Let $\Psi$ a function on the state space of the chain, with $\alpha$-tails with respect to $\pi$, $\alpha\in (0,2)$. We find sufficient conditions on the…