Related papers: On Markov--up processes and their recurrence prope…
Consider a uniformly sampled random $d$-regular graph on $n$ vertices. If $d$ is fixed and $n$ goes to $\infty$ then we can relate typical (large probability) properties of such random graph to a family of invariant random processes (called…
We formulate a criterion for the existence and uniqueness of an invariant measure for a Markov process taking values in a Polish phase space. In addition, weak-$^*$ ergodicity, that is, the weak convergence of the ergodic averages of the…
We consider estimation procedures which are recursive in the sense that each successive estimator is obtained from the previous one by a simple adjustment. We propose a wide class of recursive estimation procedures for the general…
Content of the lectures is the following. Properties of transformations equivalent to ergodicity. Birkhoff's Theorem. Properties equivalent to weak mixing. On typical properties of transformations. Lego to construct transformations. Typical…
We present a short introduction into the framework of piecewise deterministic Markov processes. We illustrate the abstract mathematical setting with a series of examples related to dispersal of biological systems, cell cycle models, gene…
These are lecture notes on the subject defined in the title. As such, they do not pretend to be really new, probably except for the only section about Poisson equations with potentials. Yet, the hope of the author is that they may serve as…
This article is a lecture note on the potential theory of (possibly non-reversible) Markov processes and on the connection of this theory with quantitative analysis of the metastability of stochastic processes.
The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
The exponential rate of convergence for some Markov operators is established. The operators correspond to continuous iterated function systems which are a very useful tool in some cell cycle models.
We consider Markov chains on partially ordered sets that generalize the success-runs and remaining life chains in reliability theory. We find conditions for recurrence and transience and give simple expressions for the invariant…
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…
We give a short overview of recent results on a specific class of Markov process: the Piecewise Deterministic Markov Processes (PDMPs). We first recall the definition of these processes and give some general results. On more specific cases…
The act of a person juggling can be viewed as a Markov process if we assume that the juggler throws to random heights. I make this association for the simplest reasonable model of random juggling and compute the steady state probabilities…
In this paper we define the notion of an open Markov process. An open Markov process is a generalization of an ordinary Markov process in which populations are allowed to flow in and out of the system at certain boundary states. We show…
In this paper, we investigate a special class of stochastic Markov processes, known as Glauber dynamics. Markov processes are importance, for example, in the study of complex systems. For this, we present the basic theory of Glauber…
We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained.…
Using the renewal approach we prove exponential inequalities for additive functionals and empirical processes of ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The inequalities…
We present a simple randomized procedure for the prediction of a binary sequence. The algorithm uses ideas from recent developments of the theory of the prediction of individual sequences. We show that if the sequence is a realization of a…
We introduce methods that allow to derive continuous-time versions of various discrete-time ergodic theorems. We then illustrate these methods by giving simple proofs and refinements of some known results as well as establishing new results…