Related papers: Quantifying Redundant Information in Predicting a …
While the terms "redundancy" and "consensus" are often used as synonyms in the context of quantum objectivity, we show here that these should be understood as two related but distinct notions, that quantify different features of the…
The rankability of data is a recently proposed problem that considers the ability of a dataset, represented as a graph, to produce a meaningful ranking of the items it contains. To study this concept, a number of rankability measures have…
We address the practical problems of estimating the information relations that characterize large networks. Building on methods developed for analysis of the neural code, we show that reliable estimates of mutual information can be obtained…
The Contextuality-by-Default approach to determining and measuring the (non)contextuality of a system of random variables requires that every random variable in the system be represented by an equivalent set of dichotomous random variables.…
We consider a natural measure of relevance: the reduction in optimal prediction risk in the presence of side information. For any given loss function, this relevance measure captures the benefit of side information for performing inference…
The partial information decomposition (PID) framework is concerned with decomposing the information that a set of (two or more) random variables (the sources) has about another variable (the target) into three types of information: unique,…
The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model…
In this paper, we evaluate dimensionality reduction methods in terms of difficulty in estimating visual information on original images from dimensionally reduced ones. Recently, dimensionality reduction has been receiving attention as the…
The redundancy principle provides the framework to study how rare events are made possible with probability 1 in accelerated time, by making many copies of similar random searchers. But what is $n$ large? To estimate large $n$ with respect…
We study the mutual information estimation for mixed-pair random variables. One random variable is discrete and the other one is continuous. We develop a kernel method to estimate the mutual information between the two random variables. The…
Knowledge of the association information between the attributes in a data set provides insight into the underlying structure of the data and explains the relationships (independence, synergy, redundancy) between the attributes and class (if…
Traditionally it had been a problem that researchers did not have access to enough spatial data to answer pressing research questions or build compelling visualizations. Today, however, the problem is often that we have too much data.…
A basic information theoretic model for summarization is formulated. Here summarization is considered as the process of taking a report of $v$ binary objects, and producing from it a $j$ element subset that captures most of the important…
In this note, we give an explicit expression for the quantile of a mixture of two random variables. We carefully examine all possible cases of discrete and continuous variables with possibly unbounded support. The result is useful for…
Prediction polling is an increasingly popular form of crowdsourcing in which multiple participants estimate the probability or magnitude of some future event. These estimates are then aggregated into a single forecast. Historically,…
One of the most fundamental questions one can ask about a pair of random variables X and Y is the value of their mutual information. Unfortunately, this task is often stymied by the extremely large dimension of the variables. We might hope…
Log-linear models are typically fitted to contingency table data to describe and identify the relationship between different categorical variables. However, the data may include observed zero cell entries. The presence of zero cell entries…
We propose a summary measure defined as the expected value of a random variable over disjoint subsets of its support that are specified by a given grid of proportions, and consider its use in a regression modeling framework. The obtained…
Estimating the dependences between random variables, and ranking them accordingly, is a prevalent problem in machine learning. Pursuing frequentist and information-theoretic approaches, we first show that the p-value and the mutual…
Measurements can be viewed as interactions between a measured system and a pointer system that imprint information about the system on the pointer. For so-called unbiased interactions, the measurement statistics--the information…