Related papers: A multiple coupon collection process and its Marko…

The successive discrete structures generated by a sequential algorithm from random input constitute a Markov chain that may exhibit long term dependence on its first few input values. Using examples from random graph theory and search…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…

This paper gives an algebraic proof of the correctness of Von Schelling formula for the probability of the coupon collector problem waiting time for non-uniform distributions and partial collections. It introduces a theorem on sums of…

Differential equations containing memory terms that depend nonlinearly on past states model a variety of non-Markovian processes. In this study, we present a Markovian embedding procedure for such equations with distributed delay by…

There has been substantial interest in developing Markov chain Monte Carlo algorithms based on piecewise-deterministic Markov processes. However existing algorithms can only be used if the target distribution of interest is differentiable…

This paper examines a discrete-time queuing system with applications to telecommunications traffic. The arrival process is a particular Markov modulated process which belongs to the class of discrete batched Markovian arrival processes. The…

Motivated by applications in bioinformatics, we consider the word collector problem, i.e. the expected number of calls to a random weighted generator of words of length $n$ before the full collection is obtained. The originality of this…

This simple note lays out a few observations which are well known in many ways but may not have been said in quite this way before. The basic idea is that when comparing two different Markov chains it is useful to couple them is such a way…

The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…

We calculate exact convergence times to reach random bipartite entanglement for various random protocols. The eigenproblem of a Markovian chain governing the process is mapped to a spin chain, thereby obtaining exact expression for the gap…

We consider Markov processes, which describe e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit…

Consider the continuous-time Markov Branching Process. In critical case we consider a situation when the generating function of intensity of transformation of particles has the infinite second moment, but its tail regularly varies in sense…

Semi-Markov processes generalize Markov processes by adding temporal memory effects as expressed by a semi-Markov kernel. We recall the path weight for a semi-Markov trajectory and the fact that thermodynamic consistency in equilibrium…

In this paper, we develop some matrix Poisson's equations satisfied by the mean and variance of the mixing time in an irreducible positive-recurrent discrete-time Markov chain with infinitely-many levels, and provide a computational…

In this paper, we consider a Markov chain choice model with single transition. In this model, customers arrive at each product with a certain probability. If the arrived product is unavailable, then the seller can recommend a subset of…

Despite the coupon collector's problem has simple probabilistic solution using inclusion/exclusion principle \cite{PolyaUrn}, starting from a particular type of recurrence differential equation it is used an analytic approach to recover…

Matrix completion aims to reconstruct a data matrix based on observations of a small number of its entries. Usually in matrix completion a single matrix is considered, which can be, for example, a rating matrix in recommendation system.…

In parameter estimation problems one computes a posterior distribution over uncertain parameters defined jointly by a prior distribution, a model, and noisy data. Markov Chain Monte Carlo (MCMC) is often used for the numerical solution of…

In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…