Related papers: A Transience Criterion for Uniformly Bounded Marko…
Estimating the transition dynamics of controlled Markov chains is crucial in fields such as time series analysis, reinforcement learning, and system exploration. Traditional non-parametric density estimation methods often assume independent…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…
This paper addresses the problem of asymptotic tracking for high-order control-affine MIMO nonlinear systems with unknown dynamic terms subject to input and transient state constraints. We introduce Barrier Integral Control (BRIC), a novel…
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous…
Nonlinear Markov chains with finite state space have been introduced in Kolokoltsov (2010). The characteristic property of these processes is that the transition probabilities do not only depend on the state, but also on the distribution of…
This paper introduces sufficient Lyapunov conditions guaranteeing exponential mean square stability of discrete-time systems with markovian delays. We provide a transformation of the discrete-time system with markovian delays into a…
This work is concerned with the stability properties of linear stochastic differential equations with random (drift and diffusion) coefficient matrices, and the stability of a corresponding random transition matrix (or exponential…
We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…
In this paper, we derive differential conditions guaranteeing the orbital stability of nonlinear hybrid limit cycles. These conditions are represented as a series of pointwise linear matrix inequalities (LMI), enabling the search for…
We obtain non-Gaussian limit laws for one-dimensional random walk in a random environment assuming that the environment is a function of a stationary Markov process. This is an extension of the work of Kesten, M. Kozlov and Spitzer for…
In a batch of synchronized queues, customers can only be serviced all at once or not at all, implying that service remains idle if at least one queue is empty. We propose that a batch of $n$ synchronized queues in a discrete-time setting is…
In this paper we show that in a multiclass Markovian network with unit rate servers, the condition that the average load $\rho$ at every server is less than unity is indeed sufficient for the stability or positive recurrence for \emph{any}…
Interest is in evaluating, by Markov chain Monte Carlo (MCMC) simulation, the expected value of a function with respect to a, possibly unnormalized, probability distribution. A general purpose variance reduction technique for the MCMC…
MeanFlow enables one-step generation in continuous spaces by learning an average velocity over a time interval rather than the instantaneous velocity field of flow matching. However, discrete state spaces do not have smooth trajectories or…
In this paper, we present new Poisson-type deviation inequalities for continuous-time Markov chains whose Wasserstein curvature or $\Gamma$-curvature is bounded below. Although these two curvatures are equivalent for Brownian motion on…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…
Conditions sufficient for the transience of the process have been established for the Markov diffusion model with switching and two modes, transient and ergodic, with intensities bounded away from zero. This paper shows limitations on the…
We assume a drift condition towards a small set and bound the mean square error of estimators obtained by taking averages along a single trajectory of a Markov chain Monte Carlo algorithm. We use these bounds to construct fixed-width…
The goal of this paper is to describe conditions which guarantee a central limit theorem for random variables, which distributions are controled by hidden Markov chains. We proved that when a Markov chain is ergodic and random variables…
We propose a simple criterion for non-Markovianity: a quantum master equation is non-Markovian if and only if its \textit{trajectory set} contains a \textit{self-intersecting trajectory} (defined in the main text). Since self-intersection…