Related papers: Ergodic Theory for Controlled Markov Chains with S…

We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then…

In this paper, a general stochastic model with controls applied at the moments when the random process hits the boundary of a given subset of the state set is proposed and studied. The general concept of the model is formulated and its…

Let $(U_n(t))_{t\in\R^d}$ be the empirical process associated to an $\R^d$-valued stationary process $(X_i)_{i\ge 0}$. We give general conditions, which only involve processes $(f(X_i))_{i\ge 0}$ for a restricted class of functions $f$,…

We propose a new definition of metastability of Markov processes on countable state spaces. We obtain sufficient conditions for a sequence of processes to be metastable. In the reversible case these conditions are expressed in terms of the…

We revisit processes generated by iterated random functions driven by a stationary and ergodic sequence. Such a process is called strongly stable if a random initialization exists, for which the process is stationary and ergodic, and for…

We consider whether ergodic Markov chains with bounded step size remain bounded in probability when their transitions are modified by an adversary on a bounded subset. We provide counterexamples to show that the answer is no in general, and…

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…

The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…

We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…

Let $(\mathcal{E},D(\mathcal{E}))$ be a quasi-regular semi-Dirichlet form and $(X_t)_{t\geq0}$ be the associated Markov process. For $u\in D(\mathcal{E})_{loc}$, denote $A_t^{[u]}:=\tilde{u}(X_{t})-\tilde{u}(X_{0})$ and…

The dissertation describes ergodic properties of some stochastic dynamical systems generated by Markov chains with values in the state space which is a Polish space. The mathematical model describing the process of cell division is…

We study ergodic quadratic optimal stochastic control problems for an affine state equation with state and control dependent noise and with stochastic coefficients. We assume stationarity of the coefficients and a finite cost condition. We…

We consider a bivariate stationary Markov chain $(X_n,Y_n)_{n\ge0}$ in a Polish state space, where only the process $(Y_n)_{n\ge0}$ is presumed to be observable. The goal of this paper is to investigate the ergodic theory and stability…

Consider a sequence $(\eta^N(t) :t\ge 0)$ of continuous-time, irreducible Markov chains evolving on a fixed finite set $E$, indexed by a parameter $N$. Denote by $R_N(\eta,\xi)$ the jump rates of the Markov chain $\eta^N_t$, and assume that…

We propose the entropy of random Markov trajectories originating and terminating at a state as a measure of the stability of a state of a Markov process. These entropies can be computed in terms of the entropy rates and stationary…

We study the problem of clustering $T$ trajectories of length $H$, each generated by one of K unknown ergodic Markov chains over a finite state space of size $S$. We derive an instance-dependent, high-probability lower bound on the…

We prove the existence of solutions for the stochastic differential equation $dX_t=b(t,X_{t-})dZ_t+a(t,X_t)dt, X_0\in\R, t\ge 0,$ with only measurable coefficients $a$ and $b$ satisfying the condition $0<\mu\le |b(t,x)|\le \nu$ and…

The first aim of this paper is to introduce a class of Markov chains on $\mathbb{Z}_+$ which are discrete self-similar in the sense that their semigroups satisfy an invariance property expressed in terms of a discrete random dilation…

This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is…

Motivated by its connection to the limit behaviour of imprecise Markov chains, we introduce and study the so-called convergence of upper transition operators: the condition that for any function, the orbit resulting from iterated…