Related papers: Annealed importance sampling for Ising models with…
We study Gibbs distributions of spins taking values in a general compact Polish space, interacting via a pair potential along the edges of a generalized random graph with a given asymptotic weight distribution $P$, obtained by annealing…
Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI)…
A new method called "variational sampling" is proposed to estimate integrals under probability distributions that can be evaluated up to a normalizing constant. The key idea is to fit the target distribution with an exponential family model…
Mutual information (MI) is a fundamental quantity in information theory and machine learning. However, direct estimation of MI is intractable, even if the true joint probability density for the variables of interest is known, as it involves…
This article considers a class of disordered mean-field combinatorial optimization problems. We focus on the Gibbs measure, where the inverse temperature does not vary with the size of the graph and the edge weights are sampled from a…
This paper considers a non-standard problem of generating samples from a low-temperature Gibbs distribution with \emph{constrained} support, when some of the coordinates of the mode lie on the boundary. These coordinates are referred to as…
We present a simple method to obtain optimal posterior distributions and improve the quality of Bayesian inference with reduced human and computational effort. Bayes' Theorem is reformulated in the language of statistical mechanics, wherein…
Variational Bayesian inference and (collapsed) Gibbs sampling are the two important classes of inference algorithms for Bayesian networks. Both have their advantages and disadvantages: collapsed Gibbs sampling is unbiased but is also…
We propose a method for the accurate estimation of rare event or failure probabilities for expensive-to-evaluate numerical models in high dimensions. The proposed approach combines ideas from large deviation theory and adaptive importance…
Importance sampling has become an important tool for the computation of tail-based risk measures. Since such quantities are often determined mainly by rare events standard Monte Carlo can be inefficient and importance sampling provides a…
Theoretical results for importance sampling rely on the existence of certain moments of the importance weights, which are the ratios between the proposal and target densities. In particular, a finite variance ensures square root convergence…
Computing the exact likelihood of data in large Bayesian networks consisting of thousands of vertices is often a difficult task. When these models contain many deterministic conditional probability tables and when the observed values are…
Bayesian inference for stochastic volatility models using MCMC methods highly depends on actual parameter values in terms of sampling efficiency. While draws from the posterior utilizing the standard centered parameterization break down…
In this paper, we consider the problem of numerical investigation of the counting statistics for a class of one-dimensional systems. Importance sampling, the cornerstone technique usually implemented for such problems, critically hinges on…
Algorithms for simulating complex physical systems or solving difficult optimization problems often resort to an annealing process. Rather than simulating the system at the temperature of interest, an annealing algorithm starts at a…
This note explores probabilistic sampling weighted by uncertainty in active learning. This method has been previously used and authors have tangentially remarked on its efficacy. The scheme has several benefits: (1) it is computationally…
We consider an evolving system for which a sequence of observations is being made, with each observation revealing additional information about current and past states of the system. We suppose each observation is made without error, but…
Importance sampling has been reported to produce algorithms with excellent empirical performance in counting problems. However, the theoretical support for its efficiency in these applications has been very limited. In this paper, we…
An anomalous mean-field solution is known to capture the non trivial phase diagram of the Ising model in annealed complex networks. Nevertheless the critical fluctuations in random complex networks remain mean-field. Here we show that a…
We propose a variant of the classical conditional gradient method for sparse inverse problems with differentiable measurement models. Such models arise in many practical problems including superresolution, time-series modeling, and matrix…