Related papers: Perfect Tempering
Markov Chain Monte Carlo (MCMC) algorithms are often used for approximate inference inside learning, but their slow mixing can be difficult to diagnose and the approximations can seriously degrade learning. To alleviate these issues, we…
Constrained decoding enables Language Models (LMs) to produce samples that provably satisfy hard constraints. However, existing constrained-decoding approaches often distort the underlying model distribution, a limitation that is especially…
Density tempering (also called density annealing) is a sequential Monte Carlo approach to Bayesian inference for general state models; it is an alternative to Markov chain Monte Carlo. When applied to state space models, it moves a…
Fast and accurate sampling method is in high demand, in order to bridge the large gaps between molecular dynamic simulations and experimental observations. Recently, integrated tempering enhanced sampling method (ITS) has been proposed and…
In this paper we study asymptotic properties of different data-augmentation-type Markov chain Monte Carlo algorithms sampling from mixture models comprising discrete as well as continuous random variables. Of particular interest to us is…
An alternative to Monte Carlo techniques requiring large sampling times is presented here. Ideas from a genetic algorithm are used to select the best initial states from many independent, parallel Metropolis-Hastings iterations that are run…
The problem of sampling constrained continuous distributions has frequently appeared in many machine/statistical learning models. Many Monte Carlo Markov Chain (MCMC) sampling methods have been adapted to handle different types of…
We explain the fundamental challenges of sampling from multimodal distributions, particularly for high-dimensional problems. We present the major types of MCMC algorithms that are designed for this purpose, including parallel tempering,…
At fine lattice spacings, Markov chain Monte Carlo simulations of QCD and other gauge theories with or without fermions are plagued by slow modes that give rise to large autocorrelation times. This can lead to simulation runs that are…
Autocorrelations in MCMC chains increase the variance of the estimators they produce. We propose the occlusion process to mitigate this problem. It is a process that sits upon an existing MCMC sampler, and occasionally replaces its samples…
An extended ensemble Monte Carlo algorithm is proposed by introducing a violation of the detailed balance condition to the update scheme of the inverse temperature in simulated tempering. Our method, irreversible simulated tempering, is…
Sampling from high-dimensional distributions is a fundamental problem in statistical research and practice. However, great challenges emerge when the target density function is unnormalized and contains isolated modes. We tackle this…
We prove bounds on the variance of a function $f$ under the empirical measure of the samples obtained by the Sequential Monte Carlo (SMC) algorithm, with time complexity depending on local rather than global Markov chain mixing dynamics.…
Bayesian data analysis is widely used across many disciplines, and representative examples in materials science include spectral analysis and sparse modeling. In such applications, the underlying models often become complex and yield…
In this paper we demonstrate that tempering Markov chain Monte Carlo samplers for Bayesian models by recursively subsampling observations without replacement can improve the performance of baseline samplers in terms of effective sample size…
We describe and analyze some Monte Carlo methods for manifolds in Euclidean space defined by equality and inequality constraints. First, we give an MCMC sampler for probability distributions defined by un-normalized densities on such…
In the present paper we identify a rigorous property of a number of tempering-based Monte Carlo sampling methods, including parallel tempering as well as partial and infinite swapping. Based on this property we develop a variety of…
Competing phases or interactions in complex many-particle systems can result in free energy barriers that strongly suppress thermal equilibration. Here we discuss how extended ensemble Monte Carlo simulations can be used to study the…
Finite element model updating is challenging because 1) the problem is oftentimes underdetermined while the measurements are limited and/or incomplete; 2) many combinations of parameters may yield responses that are similar with respect to…
Markov Chain Monte Carlo (MCMC) methods are employed to sample from a given distribution of interest, whenever either the distribution does not exist in closed form, or, if it does, no efficient method to simulate an independent sample from…