Related papers: Regeneration-enriched Markov processes with applic…
In this paper, we introduce the notion of efficiency (consistency) and examine some asymptotic properties of Markov chain Monte Carlo methods. We apply these results to the data augmentation (DA) procedure for independent and identically…
One of the most demanding calculations is to generate random samples from a specified probability distribution (usually with an unknown normalizing prefactor) in a high-dimensional configuration space. One often has to resort to using a…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
Monte Carlo methods represent the "de facto" standard for approximating complicated integrals involving multidimensional target distributions. In order to generate random realizations from the target distribution, Monte Carlo techniques use…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…
We study Markov processes conditioned so that their local time must grow slower than a prescribed function. Building upon recent work on Brownian motion with constrained local time in [5] and [33], we study transience and recurrence for a…
We propose a unified framework that extends the inference methods for classical hidden Markov models to continuous settings, where both the hidden states and observations occur in continuous time. Two different settings are analyzed: hidden…
For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of…
We present a new framework to derandomise certain Markov chain Monte Carlo (MCMC) algorithms. As in MCMC, we first reduce counting problems to sampling from a sequence of marginal distributions. For the latter task, we introduce a method…
Sequential Monte Carlo methods are a powerful framework for approximating the posterior distribution of a state variable in a sequential manner. They provide an attractive way of analyzing dynamic systems in real-time, taking into account…
The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…
The basic problem in equilibrium statistical mechanics is to compute phase space average, in which Monte Carlo method plays a very important role. We begin with a review of nonlocal algorithms for Markov chain Monte Carlo simulation in…
Monte Carlo methods are widely used importance sampling techniques for studying complex physical systems. Integrating these methods with deep learning has significantly improved efficiency and accuracy in high-dimensional problems and…
The Diffusion Monte Carlo method with constant number of walkers, also called Stochastic Reconfiguration as well as Sequential Monte Carlo, is a widely used Monte Carlo methodology for computing the ground-state energy and wave function of…
Markov chain Monte Carlo methods are a powerful tool for sampling equilibrium configurations in complex systems. One problem these methods often face is slow convergence over large energy barriers. In this work, we propose a novel method…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
We consider random walks in dynamic random environments, with an environment generated by the time-reversal of a Markov process from the oriented percolation universality class. If the influence of the random medium on the walk is small in…
This paper presents a novel theoretical Monte Carlo Markov chain procedure in the framework of graphs. It specifically deals with the construction of a Markov chain whose empirical distribution converges to a given reference one. The Markov…
We develop a modular approach to Markov chain Monte Carlo (MCMC) sampling for unnormalized target densities. In this approach, Markov chains are constructed in parallel, each constrained to a subset of the target space. The Monte Carlo…
This paper proposes a family of weighted batch means variance estimators, which are computationally efficient and can be conveniently applied in practice. The focus is on Markov chain Monte Carlo simulations and estimation of the asymptotic…