Related papers: Some things we've learned (about Markov chain Mont…

Arguing about the equilibrium distribution of continuous-time Markov chains can be vital for showing properties about the underlying systems. For example in biological systems, bistability of a chemical reaction network can hint at its…

Spreading processes are ubiquitous in natural and artificial systems. They can be studied via a plethora of models, depending on the specific details of the phenomena under study. Disease contagion and rumor spreading are among the most…

The time to converge to the steady state of a finite Markov chain can be greatly reduced by a lifting operation, which creates a new Markov chain on an expanded state space. For a class of quadratic objectives, we show an analogous behavior…

A wide class of ``counting'' problems have been studied in Computer Science. Three typical examples are the estimation of - (i) the permanent of an $n\times n$ 0-1 matrix, (ii) the partition function of certain $n-$ particle Statistical…

Statisticians often use Monte Carlo methods to approximate probability distributions, primarily with Markov chain Monte Carlo and importance sampling. Sequential Monte Carlo samplers are a class of algorithms that combine both techniques to…

The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…

This article presents several results establishing connections be- tween Markov chains and dynamical systems, from the point of view of open systems in physics. We show how all Markov chains can be understood as the information on one…

We compute the stationary distribution of a continuous-time Markov chain which is constructed by gluing together two finite, irreducible Markov chains by identifying a pair of states of one chain with a pair of states of the other and…

The main question concerns another recent advance in sequential Monte Carlo, the use of a mixture transition kernel that automatically adapts to the target distribution (Douc et al. 2006). Is there a class of static inference problems for…

This short note reviews the basic theory for quantifying both the asymptotic and preasymptotic convergence of Markov chain Monte Carlo estimators.

We devise a Monte Carlo based method for detecting whether a non-negative Markov chain is stable for a given set of parameter values. More precisely, for a given subset of the parameter space, we develop an algorithm that is capable of…

The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…

We present an intuitive, conceptual, but semi-rigorous introduction to the celebrated Markov Chain Monte Carlo method using a simple model of population dynamics as our motivation and focusing on a few elementary distributions.…

This paper is intended to appear as a chapter for the Handbook of Markov Chain Monte Carlo. The goal of this chapter is to unify various problems at the intersection of Markov chain Monte Carlo (MCMC) and machine…

For a continuous-time Markov process, we characterize the law of the first jump location when started from an arbitrary initial distribution, in terms of the invariant distribution of an auxiliary Markov process. This could be of interest…

We develop diffusion-based samplers for target distributions known up to a normalising constant. To this end, we rely on the well-known diffusion path that smoothly interpolates between a simple base distribution and the target, popularised…

These lecture notes introduce the statistical analysis of continuous-time generative models built from Markov dynamics. We begin with the stochastic-calculus foundations of score-based diffusion models, including time reversal, score…

Many systems are partially stochastic in nature. We have derived data driven approaches for extracting stochastic state machines (Markov models) directly from observed data. This chapter provides an overview of our approach with numerous…

A novel class of non-reversible Markov chain Monte Carlo schemes relying on continuous-time piecewise-deterministic Markov Processes has recently emerged. In these algorithms, the state of the Markov process evolves according to a…

The extremes of a univariate Markov chain with regulary varying stationary marginal distribution and asymptotically linear behavior are known to exhibit a multiplicative random walk structure called the tail chain. In this paper, we extend…