Related papers: Supplement to "Markov Chain Monte Carlo Based on D…
Revisions are almost entirely in the introduction and conclusion. Results are unchanged, however the comments and recommendations on different generators were changed, and more references were added.
Markov chain Monte Carlo methods are a powerful tool for sampling equilibrium configurations in complex systems. One problem these methods often face is slow convergence over large energy barriers. In this work, we propose a novel method…
Strongly Rayleigh distributions are natural generalizations of product and determinantal probability distributions and satisfy strongest form of negative dependence properties. We show that the "natural" Monte Carlo Markov Chain (MCMC) is…
This is a free textbook suitable for a one-semester course on Markov chains, covering basics of finite-state chains, many classical models, asymptotic behavior and mixing times, Monte Carlo methods, and martingales and harmonic functions.…
This paper shows how one can use Sequential Monte Carlo methods to perform what is typically done using Markov chain Monte Carlo methods. This leads to a general class of principled integration and genetic type optimization methods based on…
In this paper we investigate the continuum limits of a class of Markov chains. The investigation of such limits is motivated by the desire to model very large networks. We show that under some conditions, a sequence of Markov chains…
This text is an appendix to our work "On the growth of Kronecker coefficients", arXiv:1607.02887. Here, we provide some complementary theorems, remarks, and calculations that for the sake of space are not going to appear into the final…
Using the renewal approach we prove exponential inequalities for additive functionals and empirical processes of ergodic Markov chains, thus obtaining counterparts of inequalities for sums of independent random variables. The inequalities…
Current reporting of results based on Markov chain Monte Carlo computations could be improved. In particular, a measure of the accuracy of the resulting estimates is rarely reported. Thus we have little ability to objectively assess the…
This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises…
A large number and diversity of techniques have been offered in the literature in recent years for solving multi-label classification tasks, including classifier chains where predictions are cascaded to other models as additional features.…
This article presents several results establishing connections be- tween Markov chains and dynamical systems, from the point of view of open systems in physics. We show how all Markov chains can be understood as the information on one…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
Comment on ``Gibbs Sampling, Exponential Families and Orthogonal Polynomials'' [arXiv:0808.3852]
Convergence diagnosis for Markov chain Monte Carlo is a matter of fundamental importance in computational statistics: it determines the resources allocated to a particular sampling problem and influences the practitioner's view of the…
A new approach for optimal estimation of Markov chains with sparse transition matrices is presented.
In this note, we present few examples of Piecewise Deterministic Markov Processes and their long time behavior. They share two important features: they are related to concrete models (in biology, networks, chemistry,. . .) and they are…
Markov chain Monte Carlo is a widely-used technique for generating a dependent sequence of samples from complex distributions. Conventionally, these methods require a source of independent random variates. Most implementations use…
We introduce a stacking version of the Monte Carlo algorithm in the context of option pricing. Introduced recently for aeronautic computations, this simple technique, in the spirit of current machine learning ideas, learns control variates…
This article is a pedagogical review of Monte Carlo methods for the self-avoiding walk, with emphasis on the extraordinarily efficient algorithms developed over the past decade. Many more details can be found in hep-lat/9405016.