Related papers: Asymptotically linear iterated function systems on…

The upper extremes of a Markov chain with regulary varying stationary marginal distribution are known to exhibit under general conditions a multiplicative random walk structure called the tail chain. More generally, if the Markov chain is…

We review the theory of regenerative processes, which are processes that can be intuitively seen as comprising of i.i.d.\ cycles. Although we focus on the classical definition, we present a more general definition that allows for some form…

We propose certain conditions which are sufficient for the functional law of the iterated logarithm (the Strassen invariance principle) for some general class of non-stationary Markov-Feller chains. This class may be briefly specified by…

We obtain complementary recurrence and transience criteria for processes $X=(X_n)_{n \ge 0}$ with values in $\mathbb R^d_+$ fulfilling a non-linear equation $X_{n+1}=MX_n+g(X_n)+ \xi_{n+1}$. Here $M$ denotes a primitive matrix having…

We prove a complete class theorem that characterizes \emph{all} stationary time reversible Markov processes whose finite dimensional marginal distributions (of all orders) are infinitely divisible. Aside from two degenerate cases (iid and…

Consider a Markov chain defined on a finite state space, X, that leaves invariant the uniform distribution on X, and whose transition probabilities are integer multiples of 1/Q, for some integer Q. I show how a simulation of n transitions…

We consider a Markov chain on $R^+$ with asymptotically zero drift and finite second moments of jumps which is positive recurrent. A power-like asymptotic behaviour of the invariant tail distribution is proven; such a heavy-tailed invariant…

This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a…

Consider an N-dimensional Markov chain obtained from N one-dimensional random walks by Doob h-transform with the q-Vandermonde determinant. We prove that as N becomes large, these Markov chains converge to an infinite-dimensional Feller…

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…

Classical linear regression is considered for a case when regression parameters depend on the external random environment. The last is described as a continuous time Markov chain with finite state space. Here the expected sojourn times in…

We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.}…

We introduce a general algorithm for the computation of the scale functions of a spectrally negative L\'evy process $X$, based on a natural weak approximation of $X$ via upwards skip-free continuous-time Markov chains with stationary…

Consider a first-order autoregressive process $X_i=\beta X_{i-1}+\varepsilon_i,$ where $\varepsilon_i=G(\eta_i,\eta_{i-1},\ldots)$ and $\eta_i,i\in\mathbb{Z}$ are i.i.d. random variables. Motivated by two important issues for the inference…

The Law of the Iterated Logarithm for some Markov operators, which converge exponentially to the invariant measure, is established. The operators correspond to iterated function systems which, for example, may be used to generalize the cell…

We consider the random reversible Markov kernel K obtained by assigning i.i.d. nonnegative weights to the edges of the complete graph over n vertices and normalizing by the corresponding row sum. The weights are assumed to be in the domain…

Stochastic resetting describes dynamics which are reinitialized to a reference state at random times. These protocols are attracting significant interest: they can stabilize nonequilibrium stationary states, generate correlations in…

Continuous-time Markov chains on non-negative integers can be used for modeling biological systems, population dynamics, and queueing models. Qualitative behaviors of birth-and-death models, typical examples of such one-dimensional…

Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…

A rigorous and largely self-contained account of (a) the bread-and-butter concepts and techniques in Markov chain theory and (b) the long-term behaviour of chains. As much as possible, the treatment is probabilistic instead of analytical (I…