Related papers: Mutual information, matroids and extremal dependen…
The mutual information of two random variables i and j with joint probabilities t_ij is commonly used in learning Bayesian nets as well as in many other fields. The chances t_ij are usually estimated by the empirical sampling frequency…
This article introduces the Multidimensional Research Assessment Matrix of scientific output. Its base notion holds that the choice of metrics to be applied in a research assessment process depends upon the unit of assessment, the research…
In the setting where we ask participants multiple similar possibly subjective multi-choice questions (e.g. Do you like Bulbasaur? Y/N; do you like Squirtle? Y/N), peer prediction aims to design mechanisms that encourage honest feedback…
One of the main notions of information theory is the notion of mutual information in two messages (two random variables in Shannon information theory or two binary strings in algorithmic information theory). The mutual information in $x$…
In this paper, we propose a new mutual information framework for multi-agent reinforcement learning to enable multiple agents to learn coordinated behaviors by regularizing the accumulated return with the simultaneous mutual information…
The selection of features that are relevant for a prediction or classification problem is an important problem in many domains involving high-dimensional data. Selecting features helps fighting the curse of dimensionality, improving the…
The mutual information method has demonstrated to be very useful for deriving the potential order parameter of a system. Although the method suggests some constraints which help to define this quantity, there is still some freedom in the…
We propose learning discrete structured representations from unlabeled data by maximizing the mutual information between a structured latent variable and a target variable. Calculating mutual information is intractable in this setting. Our…
In this survey, we present and compare different approaches to estimate Mutual Information (MI) from data to analyse general dependencies between variables of interest in a system. We demonstrate the performance difference of MI versus…
Feature selection problems arise in a variety of applications, such as microarray analysis, clinical prediction, text categorization, image classification and face recognition, multi-label learning, and classification of internet traffic.…
Kolmogorov's representation theorem provides a framework for decomposing any arbitrary real-valued, multivariate, and continuous function into a two-layer nested superposition of a finite number of functions. The functions at these two…
Forecasting techniques for assessing the power of future experiments to discriminate between theories or discover new laws of nature are of great interest in many areas of science. In this paper, we introduce a Bayesian forecasting method…
In a secret sharing scheme, shares of a secret are distributed to participants in such a way that only certain predetermined sets of participants are qualified to reconstruct the secret. An access structure on a set of participants…
Correlations disguised in various forms underlie a host of important phenomena in classical and quantum systems, such as information and energy exchanges. The quantum mutual information and the norm of the correlation matrix are both…
Analogical reasoning depends fundamentally on the ability to learn and generalize about relations between objects. We develop an approach to relational learning which, given a set of pairs of objects…
In a previous report we have evaluated analytically the mutual information between the firing rates of N independent units and a set of multi-dimensional continuous+discrete stimuli, for a finite population size and in the limit of large…
The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that based on mutual information, for a class of structured spiking neuronal network. In…
The work is devoted to study of quantum mutual information and coherent information -- the two important characteristics of quantum communication channel. Appropriate definitions of these quantities in the infinite-dimensional case are…
We use a well known model (T. Vicsek et al. Phys Rev Lett 15, 1226 (1995)) for flocking to test mutual information as a tool for detecting order-disorder transitions, in particular when observations of the system are limited. We show that…
We consider the problem of decomposing the total mutual information conveyed by a pair of predictor random variables about a target random variable into redundant, unique and synergistic contributions. We focus on the relationship between…