Related papers: Robust estimation of a Markov chain transition mat…

We study a variable length Markov chain model associated with a group of stationary processes that share the same context tree but each process has potentially different conditional probabilities. We propose a new model selection and…

A common tool in the practice of Markov Chain Monte Carlo is to use approximating transition kernels to speed up computation when the desired kernel is slow to evaluate or intractable. A limited set of quantitative tools exist to assess the…

The paper presents efficient approaches for evaluating convergence rate in total variation for finite and general linear Markov chains. The motivation for studying convergence rate in this metric is its usefulness in various limit theorems.…

Metastability is a physical phenomenon ubiquitous in first order phase transitions. A fruitful mathematical way to approach this phenomenon is the study of rare transitions Markov chains. For Metropolis chains associated with Statistical…

The problem of estimating an unknown discrete distribution from its samples is a fundamental tenet of statistical learning. Over the past decade, it attracted significant research effort and has been solved for a variety of divergence…

A novel class of non-reversible Markov chain Monte Carlo schemes relying on continuous-time piecewise-deterministic Markov Processes has recently emerged. In these algorithms, the state of the Markov process evolves according to a…

The application of the Markov chain to modeling agricultural succession is well known. In most cases, the main problem is the inference of the model, i.e. the estimation of the transition matrix. In this work we present methods to estimate…

We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…

Many economic models feature monotone Markov dynamics on state spaces that may be noncompact. Establishing existence, uniqueness, and stability of stationary distributions in such settings has required a patchwork of sufficient conditions,…

Markov chain Monte Carlo is a widely-used technique for generating a dependent sequence of samples from complex distributions. Conventionally, these methods require a source of independent random variates. Most implementations use…

We propose a new Markov chain Monte Carlo method in which trial configurations are generated by evolving a state, sampled from a prior distribution, using a Markov transition matrix. We present two prototypical algorithms and derive their…

This article surveys recent advancements of strategy designs for persistent robotic surveillance tasks with the focus on stochastic approaches. The problem describes how mobile robots stochastically patrol a graph in an efficient way where…

For many Markov chains of practical interest, the invariant distribution is extremely sensitive to perturbations of some entries of the transition matrix, but insensitive to others; we give an example of such a chain, motivated by a problem…

The Markov Chain Monte Carlo method is the dominant paradigm for posterior computation in Bayesian analysis. It is common to control computation time by making approximations to the Markov transition kernel. Comparatively little attention…

We are studying stationary random processes with conditional polynomial moments that allow a continuous path modification. Processes with continuous path modification, are important because they are relatively easy to simulate. One does not…

In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…

In any Markov chain with finite state space the distribution of transition records always belongs to the exponential family. This observation is used to prove a fluctuation theorem, and to show that the dynamical entropy of a stationary…

"Quantum trajectories" are solutions of stochastic differential equations also called Belavkin or Stochastic Schr\"odinger Equations. They describe random phenomena in quantum measurement theory. Two types of such equations are usually…

We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…

In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…