Related papers: A new criterion based on Kullback-Leibler informat…
Acquiring information on spatial phenomena can be costly and time-consuming. In this context, to obtain reliable global knowledge, the choice of measurement location is a crucial issue. Space-lling designs are often used to control…
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on…
Optimal experimental design seeks to determine the most informative allocation of experiments to infer an unknown statistical quantity. In this work, we investigate the optimal design of experiments for {\em estimation of linear functionals…
We view sequential design as a model selection problem to determine which new observation is expected to be the most informative, given the existing set of observations. For estimating a probability distribution on a bounded interval, we…
A common failure mode of density models trained as variational autoencoders is to model the data without relying on their latent variables, rendering these variables useless. Two contributing factors, the underspecification of the model and…
Feature selection and reducing the dimensionality of data is an essential step in data analysis. In this work, we propose a new criterion for feature selection that is formulated as conditional information between features given the labeled…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
In this paper we discuss new adaptive proposal strategies for sequential Monte Carlo algorithms--also known as particle filters--relying on criteria evaluating the quality of the proposed particles. The choice of the proposal distribution…
The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…
This chapter discusses a general design approach to planning computer experiments, which seeks design points that fill a bounded design region as uniformly as possible. Such designs are broadly referred to as space-filling designs.
Optimal dimensionality reduction methods are proposed for the Bayesian inference of a Gaussian linear model with additive noise in presence of overabundant data. Three different optimal projections of the observations are proposed based on…
The selection of optimal designs for generalized linear mixed models is complicated by the fact that the Fisher information matrix, on which most optimality criteria depend, is computationally expensive to evaluate. Our focus is on the…
Scientific modeling applications often require estimating a distribution of parameters consistent with a dataset of observations - an inference task also known as source distribution estimation. This problem can be ill-posed, however, since…
The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for…
We introduce the notion of relative volume entropy for two spacetimes with preferred compact spacelike foliations. This is accomplished by applying the notion of Kullback-Leibler divergence to the volume elements induced on spacelike…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
For a parametric model of distributions, the closest distribution in the model to the true distribution located outside the model is considered. Measuring the closeness between two distributions with the Kullback-Leibler (K-L) divergence,…
Bayesian optimal experimental design (OED) provides a principled framework for selecting observations or experiments. We introduce new Bayesian design criteria based on the expected Wasserstein-$p$ distance between the prior and posterior…
In this paper we tackle the problem of generating uniform designs in very small subregions of computer model input space that have been identified in previous experiments as worthy of further study. The method is capable of producing…
Analysis of a probabilistic system often requires to learn the joint probability distribution of its random variables. The computation of the exact distribution is usually an exhaustive precise analysis on all executions of the system. To…