Related papers: Asymptotically linear iterated function systems on…
Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete time, to Markov renewal processes and Markov chains in continuous time are discussed. Three alternative Kemeny's functions and their variants are…
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…
Empirical processes for stationary, causal sequences are considered. We establish empirical central limit theorems for classes of indicators of left half lines, absolutely continuous functions and piecewise differentiable functions. Sample…
The tail chain of a Markov chain can be used to model the dependence between extreme observations. For a positive recurrent Markov chain, the tail chain aids in describing the limit of a sequence of point processes $\{N_n,n\geq1\}$,…
We consider stochastic dynamical systems on ${\mathbb{R}}$, that is, random processes defined by $X_n^x=\Psi_n(X_{n-1}^x)$, $X_0^x=x$, where $\Psi _n$ are i.i.d. random continuous transformations of some unbounded closed subset of…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
An asymptotic model for extreme behavior of certain Markov chains is the "tail chain". Generally taking the form of a multiplicative random walk, it is useful in deriving extremal characteristics such as point process limits. We place this…
In this study, a new extension of the Markov Renewal theory is introduced by allowing time to evolve in multiple dimensions. The resulting chains are referred to as multi-time Markov Renewal chains and since this extension is new, the state…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
We consider a stochastic recurrence equation of the form $Z_{n+1} = A_{n+1} Z_n+B_{n+1}$, where $\mathbb{E}[\log A_1]<0$, $\mathbb{E}[\log^+ B_1]<\infty$ and $\{(A_n,B_n)\}_{n\in\mathbb{N}}$ is an i.i.d. sequence of positive random vectors.…
We consider linear iterated function systems with a random multiplicative error on the real line. Our system is $\{x\mapsto d_i + \lambda_i Y x\}_{i=1}^m$, where $d_i\in \R$ and $\lambda_i>0$ are fixed and $Y> 0$ is a random variable with…
Lyapunov functions are fundamental to establishing the stability of Markovian models, yet their construction typically demands substantial creativity and analytical effort. In this paper, we show that deep learning can automate this process…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
In this paper we study the asymptotic behavior of linear processes having as innovations mean zero, square integrable functions of stationary reversible Markov chains. In doing so we shall preserve the generality of coefficients assuming…
Consider a Markov chain $\{X_n\}_{n\ge 0}$ with an ergodic probability measure $\pi$. Let $\Psi$ a function on the state space of the chain, with $\alpha$-tails with respect to $\pi$, $\alpha\in (0,2)$. We find sufficient conditions on the…
The dynamics in games involving multiple players, who adaptively learn from their past experience, is not yet well understood. We analyzed a class of stochastic games with Markov strategies in which players choose their actions…
This paper investigates tail asymptotics of stationary distributions and quasi-stationary distributions (QSDs) of continuous-time Markov chains on subsets of the non-negative integers. Based on the so-called flux-balance equation, we…
We study the stochastic dynamics of a system of interacting species in a stochastic environment by means of a continuous-time Markov chain with transition rates depending on the state of the environment. Models of gene regulation in systems…
We extend Goldie's implicit renewal theorem to the arithmetic case, which allows us to determine the tail behavior of the solution of various random fixed point equations. It turns out that the arithmetic and nonarithmetic cases are very…
The analysis of many problems of interest associated with Markov chains, e.g. stationary distributions, moments of first passage time distributions and moments of occupation time random variables, involves the solution of a system of linear…