Related papers: Distribution of Mutual Information
When prior information is lacking, the go-to strategy for probabilistic inference is to combine a "default prior" and the likelihood via Bayes's theorem. Objective Bayes, (generalized) fiducial inference, etc. fall under this umbrella. This…
Shannon information was defined for characterizing the uncertainty information of classical probabilistic distributions. As an uncertainty measure it is generally believed to be positive. This holds for any information quantity from two…
The statistical inverse problem of estimating the probability distribution of an infinite-dimensional unknown given its noisy indirect observation is studied in the Bayesian framework. In practice, one often considers only…
Fields like public health, public policy, and social science often want to quantify the degree of dependence between variables whose relationships take on unknown functional forms. Typically, in fact, researchers in these fields are…
Structure and parameters in a Bayesian network uniquely specify the probability distribution of the modeled domain. The locality of both structure and probabilistic information are the great benefits of Bayesian networks and require the…
Multivariate pattern analyses approaches in neuroimaging are fundamentally concerned with investigating the quantity and type of information processed by various regions of the human brain; typically, estimates of classification accuracy…
We exploit qualitative probabilistic relationships among variables for computing bounds of conditional probability distributions of interest in Bayesian networks. Using the signs of qualitative relationships, we can implement abstraction…
In this paper reference and probability-matching priors are derived for the univariate Student $t$-distribution. These priors generally lead to procedures with properties frequentists can relate to while still retaining Bayes validity. The…
Which neural networks are similar is a fundamental question for both machine learning and neuroscience. Here, it is proposed to base comparisons on the predictive distributions of linear readouts from intermediate representations. In…
We develop a result on expected posteriors for Bayesians with heterogenous priors, dubbed information validates the prior (IVP). Under familiar ordering requirements, Anne expects a (Blackwell) more informative experiment to bring Bob's…
The proposed approach extends the confidence posterior distribution to the semi-parametric empirical Bayes setting. Whereas the Bayesian posterior is defined in terms of a prior distribution conditional on the observed data, the confidence…
We describe algorithms for learning Bayesian networks from a combination of user knowledge and statistical data. The algorithms have two components: a scoring metric and a search procedure. The scoring metric takes a network structure,…
Partial information decomposition (PID) of the multivariate mutual information describes the distinct ways in which a set of source variables contains information about a target variable. The groundbreaking work of Williams and Beer has…
In public opinion studies, the relationships between opinions on different topics are likely to shift based on the characteristics of the respondents. Thus, understanding the complexities of public opinion requires methods that can account…
Unit-level models for survey data offer many advantages over their area-level counterparts, such as potential for more precise estimates and a natural benchmarking property. However two main challenges occur in this context: accounting for…
The asymptotic mutual information (MI) analysis for multiple-input multiple-output (MIMO) systems over double-scattering channels has achieved engaging results, but the convergence rates of the mean, variance, and the distribution of the MI…
We propose a Bayesian inference framework to estimate uncertainties in inverse scattering problems. Given the observed data, the forward model and their uncertainties, we find the posterior distribution over a finite parameter field…
The problem of calculating the rate of mutual information between two coarse-grained variables that together specify a continuous time Markov process is addressed. As a main obstacle, the coarse-grained variables are in general…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…
The widely applicable information criterion (WAIC) has been used as a model selection criterion for Bayesian statistics in recent years. It is an asymptotically unbiased estimator of the Kullback-Leibler divergence between a Bayesian…