Related papers: Information gains from Monte Carlo Markov Chains
Rich data generating mechanisms are ubiquitous in this age of information and require complex statistical models to draw meaningful inference. While Bayesian analysis has seen enormous development in the last 30 years, benefitting from the…
One of the well-known challenges in optimal experimental design is how to efficiently estimate the nested integrations of the expected information gain. The Gaussian approximation and associated importance sampling have been shown to be…
It is widely known that the performance of Markov chain Monte Carlo (MCMC) can degrade quickly when targeting computationally expensive posterior distributions, such as when the sample size is large. This has motivated the search for MCMC…
The necessary information to distinguish a local inhomogeneous mass density field from its spatial average on a compact domain of the universe can be measured by relative information entropy. The Kullback-Leibler (KL) formula arises very…
This article reviews the application of advanced Monte Carlo techniques in the context of Multilevel Monte Carlo (MLMC). MLMC is a strategy employed to compute expectations which can be biased in some sense, for instance, by using the…
Markov chain Monte Carlo (MCMC) methods provide powerful framework for sampling unknown probability measures across a wide range of scientific applications. In some settings, the target distribution is supported on a lower-dimensional…
We introduce a generalised micro-macro Markov chain Monte Carlo (mM-MCMC) method with pseudo-marginal approximation to the free energy, that is able to accelerate sampling of the microscopic Gibbs distributions when there is a time-scale…
Adaptive and interacting Markov chain Monte Carlo algorithms (MCMC) have been recently introduced in the literature. These novel simulation algorithms are designed to increase the simulation efficiency to sample complex distributions.…
We rely on Monte Carlo (MC) simulations to interpret searches for new physics at the Large Hadron Collider (LHC) and elsewhere. These simulations result in noisy and approximate estimators of selection efficiencies and likelihoods. In this…
We propose a method to efficiently integrate truncated probability densities. The method uses Markov chain Monte Carlo method to sample from a probability density matching the function being integrated. The required normalisation or…
In many computational problems, using the Markov Chain Monte Carlo (MCMC) can be prohibitively time-consuming. We propose MCMC-Net, a simple yet efficient way to accelerate MCMC via neural networks. The key idea of our approach is to…
We formulate both Markov chain Monte Carlo (MCMC) sampling algorithms and basic statistical physics in terms of elementary symmetries. This perspective on sampling yields derivations of well-known MCMC algorithms and a new parallel…
The information detection of complex systems from data is currently undergoing a revolution, driven by the emergence of big data and machine learning methodology. Discovering governing equations and quantifying dynamical properties of…
The literature in social network analysis has largely focused on methods and models which require complete network data; however there exist many networks which can only be studied via sampling methods due to the scale or complexity of the…
An important feature of Bayesian statistics is the opportunity to do sequential inference: the posterior distribution obtained after seeing a dataset can be used as prior for a second inference. However, when Monte Carlo sampling methods…
Analyzes of next-generation galaxy data require accurate treatment of systematic effects such as the bias between observed galaxies and the underlying matter density field. However, proposed models of the phenomenon are either numerically…
We consider the recently introduced Transformation-based Markov Chain Monte Carlo (TMCMC) (Dutta and Bhattacharya (2014)), a methodology that is designed to update all the parameters simultaneously using some simple deterministic…
Recent likelihood theory produces $p$-values that have remarkable accuracy and wide applicability. The calculations use familiar tools such as maximum likelihood values (MLEs), observed information and parameter rescaling. The usual…
The Shannon entropy, and related quantities such as mutual information, can be used to quantify uncertainty and relevance. However, in practice, it can be difficult to compute these quantities for arbitrary probability distributions,…
In this work we introduce a method for estimating entropy rate and entropy production rate from finite symbolic time series. From the point of view of statistics, estimating entropy from a finite series can be interpreted as a problem of…