Related papers: Ergodic Theory for Controlled Markov Chains with S…
In many applications, the common assumption that a driving noise process affecting a system is independent or Markovian may not be realistic, but the noise process may be assumed to be stationary. To study such problems, this paper…
We study a $d$-dimensional stochastic process $\mathbf{X}$ which arises from a L\'evy process $\mathbf{Y}$ by partial resetting, that is the position of the process $\mathbf{X}$ at a Poisson moment equals $c$ times its position right before…
A common tool in the practice of Markov Chain Monte Carlo is to use approximating transition kernels to speed up computation when the desired kernel is slow to evaluate or intractable. A limited set of quantitative tools exist to assess the…
The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria,…
We study ergodic properties of Markovian multiclass many-server queues which are uniform over scheduling policies, as well as the size n of the system. The system is heavily loaded in the Halfin-Whitt regime, and the scheduling policies are…
We present an approach for testing for the existence of continuous generators of discrete stochastic transition matrices. Typically, the known approaches to ascertain the existence of continuous Markov processes are based in the assumption…
Let $P$ be the transition matrix of a finite, irreducible and reversible Markov chain. We say the continuous time Markov chain $X$ has transition matrix $P$ and speed $\lambda$ if it jumps at rate $\lambda$ according to the matrix $P$. Fix…
This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…
We prove that the class of discrete time stationary max-stable process satisfying the Markov property is equal, up to time reversal, to the class of stationary max-autoregressive processes of order $1$. A similar statement is also proved…
In this article, we study the ergodic risk-sensitive control problem for controlled regime-switching diffusions. Under a blanket stability hypothesis, we solve the associated nonlinear eigenvalue problem for weakly coupled systems and…
We investigate the convergence in distribution of sequential empirical processes of dependent data indexed by a class of functions F. Our technique is suitable for processes that satisfy a multiple mixing condition on a space of functions…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…
This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a…
We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled…
We study discrete time Markov processes with periodic or open boundary conditions and with inhomogeneous rates in the bulk. The Markov matrices are given by the inhomogeneous transfer matrices introduced previously to prove the…
We construct a stationary Markov process with trivial tail sigma-field and a nondegenerate observation process such that the corresponding nonlinear filtering process is not uniquely ergodic. This settles in the negative a conjecture of the…
We consider a Markov process in continuous time with a finite number of discrete states. The time-dependent probabilities of being in any state of the Markov chain are governed by a set of ordinary differential equations, whose dimension…
We study a stochastic version of the classical Becker-D\"oring model, a well-known kinetic model for cluster formation that predicts the existence of a long-lived metastable state before a thermodynamically unfavorable nucleation occurs,…
Ergodicity of random dynamical systems with a periodic measure is obtained on a Polish space. In the Markovian case, the idea of Poincar\'e sections is introduced. It is proved that if the periodic measure is PS-ergodic, then it is ergodic.…
Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or enhanced sampling methods may improve the…