Related papers: Information gains from Monte Carlo Markov Chains
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
Markov chain Monte Carlo (MCMC) methods generate samples that are asymptotically distributed from a target distribution of interest as the number of iterations goes to infinity. Various theoretical results provide upper bounds on the…
In the context of Bayesian inversion for scientific and engineering modeling, Markov chain Monte Carlo sampling strategies are the benchmark due to their flexibility and robustness in dealing with arbitrary posterior probability density…
We propose a multilevel Markov chain Monte Carlo (MCMC) method for the Bayesian inference of random field parameters in PDEs using high-resolution data. Compared to existing multilevel MCMC methods, we additionally consider level-dependent…
In Markov Chain Monte Carlo (MCMC) simulations, the thermal equilibria quantities are estimated by ensemble average over a sample set containing a large number of correlated samples. These samples are selected in accordance with the…
We propose a generic Markov Chain Monte Carlo (MCMC) algorithm to speed up computations for datasets with many observations. A key feature of our approach is the use of the highly efficient difference estimator from the survey sampling…
Monte Carlo and Quasi-Monte Carlo methods present a convenient approach for approximating the expected value of a random variable. Algorithms exist to adaptively sample the random variable until a user defined absolute error tolerance is…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
Information theory has been taken as a prospective tool for quantifying the complexity of complex networks. In this paper, we first study the information entropy or uncertainty of a path using the information theory. Then we apply the path…
The widespread use of Markov Chain Monte Carlo (MCMC) methods for high-dimensional applications has motivated research into the scalability of these algorithms with respect to the dimension of the problem. Despite this, numerous problems…
Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is…
Inference after model selection presents computational challenges when dealing with intractable conditional distributions. Markov chain Monte Carlo (MCMC) is a common method for sampling from these distributions, but its slow convergence…
The reversible jump Markov chain Monte Carlo (RJMCMC) method offers an across-model simulation approach for Bayesian estimation and model comparison, by exploring the sampling space that consists of several models of possibly varying…
For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…
In this paper we consider the modeling of measurement error for fund returns data. In particular, given access to a time-series of discretely observed log-returns and the associated maximum over the observation period, we develop a…
The interpretation of cosmological observables requires the use of increasingly sophisticated theoretical models. Since these models are becoming computationally very expensive and display non-trivial uncertainties, the use of standard…
A randomized algorithm for finding sparse cuts is given which is based on constructing a dual markov chain called multiscale rings process(MRP) and a new concept of entropy. It is shown how the time to absorption of the dual process…
A scheme is presented to extract detailed dynamical signatures from successive measurements of complex systems. Relative entropy based time series tools are used to quantify the gain in predictive power of increasing past knowledge. By…
Markov chain Monte Carlo (MCMC) is a powerful tool for sampling from complex probability distributions. Despite its versatility, MCMC often suffers from strong autocorrelation and the negative sign problem, leading to slowing down the…
Monte Carlo algorithms often aim to draw from a distribution $\pi$ by simulating a Markov chain with transition kernel $P$ such that $\pi$ is invariant under $P$. However, there are many situations for which it is impractical or impossible…