Related papers: Correct Approximation of Stationary Distributions

For downward skip-free continuous-time Markov chains on non-negative integers stopped at zero, existence of a quasi-stationary distribution is studied. The scale function for these processes is introduced and the boundary is classified by a…

In this invited contribution, we revisit the stochastic shortest path problem, and show how recent results allow one to improve over the classical solutions: we present algorithms to synthesize strategies with multiple guarantees on the…

The problem of sampling from the stationary distribution of a Markov chain finds widespread applications in a variety of fields. The time required for a Markov chain to converge to its stationary distribution is known as the classical…

Perfect sampling is a technique that uses coupling arguments to provide a sample from the stationary distribution of a Markov chain in a finite time without ever computing the distribution. This technique is very efficient if all the events…

In this letter we announce rigorous results that elucidate the relation between metastable states and low-lying eigenvalues in Markov chains in a much more general setting and with considerable greater precision as was so far available.…

In this work, we study how to efficiently obtain perfect samples from a discrete distribution $\mathcal{D}$ given access only to pairwise comparisons of elements of its support. Specifically, we assume access to samples $(x, S)$, where $S$…

We propose a sequential Markov chain Monte Carlo (SMCMC) algorithm to sample from a sequence of probability distributions, corresponding to posterior distributions at different times in on-line applications. SMCMC proceeds as in usual MCMC…

This article describes a method for computing limits of a class of non-stationary Markov chains motivated by healthcare sojourn-time cycles. A mathematical validation of the computation method is also given. Applications are described that…

This paper uses the generator approach of Stein's method to analyze the gap between steady-state distributions of Markov chains and diffusion processes. Until now, the standard way to invoke Stein's method for this problem was to use the…

Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies…

In a recent paper, Shah [arXiv:2502.03073] derived an explicit expression for the distribution of occupancy times for a two-state Markov chain, using a method based on enumerating sample paths. We consider here the more general problem of…

For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of…

Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…

In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…

We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…

Cross-sectional observations from a dynamical system can be modeled via steady-state distributions of Markov processes. The major challenge is then to determine whether the process parameters can be identified and estimated from the…

We develop a novel class of MCMC algorithms based on a stochastized Nesterov scheme. With an appropriate addition of noise, the result is a time-inhomogeneous underdamped Langevin equation, which we prove emits a specified target…

We propose two algorithms for simulating continuous time Markov chains in the presence of metastability. We show that the algorithms correctly estimate, under the ergodicity assumption, stationary averages of the process. Both algorithms,…

Consider the standard stochastic reaction network model where the dynamics is given by a continuous-time Markov chain over a discrete lattice. For such models, estimation of parameter sensitivities is an important problem, but the existing…

Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…