Related papers: Cyclic products and optimal traps in cyclic birth …

We present a learning model predictive control (MPC) scheme for chance-constrained Markov jump systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are…

We consider Markov decision processes (MDP) as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in a single state, or in a given set of…

The use of coordinate processes for the modelling of impulse control for general Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of…

Consider the following Markov chain on permutations of length $n$. At each time step we choose a random position. If the letter at that position is smaller than the letter immediately to the left (cyclically) then these letters swap…

In a first part, we prove a Lyapunov-type criterion for the $\xi\_1$-positive recurrence of absorbed birth and death processes and provide new results on the domain of attraction of the minimal quasi-stationary distribution. In a second…

A cyclic ordering of the points in a Mendelsohn triple system of order $v$ (or MTS$(v)$) is called a sequencing. A sequencing $D$ is $\ell$-good if there does not exist a triple $(x,y,z)$ in the MTS$(v)$ such that (1) the three points…

Consideration is given to the three different analytical methods for the computation of upper bounds for the rate of convergence to the limiting regime of one specific class of (in)homogeneous continuous-time Markov chains. This class is…

Let $\mathcal{A}(R,S)$ denote the class of all matrices of zeros and ones with row sum vector $R$ and column sum vector~$S$. We introduce the notion of an inversion in a $(0,1)$--matrix. This definition extends the standard notion of an…

We consider a supercritical general branching population where the lifetimes of individuals are i.i.d. with arbitrary distribution and each individual gives birth to new individuals at Poisson times independently from each others. The…

It is shown that a seemingly harmless reordering of the steps in a block Gibbs sampler can actually invalidate the algorithm. In particular, the Markov chain that is simulated by the "out-of-order" block Gibbs sampler does not have the…

Let us consider a homogeneous Markov chain with discrete time and with a finite set of states $E_0,\ldots,E_n$ such that the state $E_0$ is absorbing, states $E_1,\ldots,E_n$ are nonrecurrent. The goal of this work is to study frequencies…

Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stopping times $\lambda_n$…

Spatial birth and death processes are obtained as solutions of a system of stochastic equations. The processes are required to be locally finite, but may involve an infinite population over the full (noncompact) type space. Conditions are…

A random sequence having two segments being the homogeneous Markov processes is registered. Each segment has his own transition probability law and the length of the segment is unknown and random. The transition probabilities of each…

Birth-death processes track the size of a univariate population, but many biological systems involve interaction between populations, necessitating models for two or more populations simultaneously. A lack of efficient methods for…

We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pm\gamma$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases:…

We show that the transition probability of the Markoc chain $(G(j,1),...,G(j,n))_{j\ge 1}$, where the $G(i,j)'s$ are certain directed last-passage times, is given by a determinant of a special form. An analogous formula has recently been…

Suppose that X_n, n>=0 is a stationary Markov chain and V is a certain function on a phase space of the chain, called an observanle. We say that the observable satisfies the central limit theorem (C.L.T.) if Y_n:=N^{-1/2}\sum_{n=0}^NV(X_n)…

We study the rate of convergence of the Markov chain on $S_n$ which starts with a random $(n-k)$-cycle for a fixed $k \geq 1$, followed by random transpositions. The convergence to the stationary distribution turns out to be of order $n$.…

In this paper, we study a biased version of the nearest-neighbor transposition Markov chain on the set of permutations where neighboring elements $i$ and $j$ are placed in order $(i,j)$ with probability $p_{i,j}$. Our goal is to identify…