Related papers: Remarks on the Most Informative Function Conjectur…

By combining a bound on the absolute value of the difference of mutual information between two joint probablity distributions with a fixed variational distance, and a bound on the probability of a maximal deviation in variational distance…

We address the problem of continuous-variable quantum phase estimation in the presence of linear disturbance at the Hamiltonian level, by means of Gaussian probe states. In particular we discuss both unitary and random disturbance, by…

Normalized mutual information is widely used as a similarity measure for evaluating the performance of clustering and classification algorithms. In this paper, we argue that results returned by the normalized mutual information are biased…

The invariant response was defined from a formulation of the fluctuation-response theorem in the space of probability distributions. An inequality is here conjectured which sets the mutual information as an upper bound to the invariant…

Let $T_{\epsilon}$ be the noise operator acting on Boolean functions $f:\{0, 1\}^n\to \{0, 1\}$, where $\epsilon\in[0, 1/2]$ is the noise parameter. Given $\alpha>1$ and fixed mean $\mathbb{E} f$, which Boolean function $f$ has the largest…

Inspired by Hilberg's hypothesis, which states that mutual information between blocks for natural language grows like a power law, we seek for links between power-law growth rate of algorithmic mutual information and of some estimator of…

We consider the problem of estimating the direction of arrival of a signal embedded in $K$-distributed noise, when secondary data which contains noise only are assumed to be available. Based upon a recent formula of the Fisher information…

We show that part I of uniform Martin's conjecture follows from a local phenomenon, namely that if a non-constant Turing invariant function goes from the Turing degree $\boldsymbol x$ to the Turing degree $\boldsymbol y$, then $\boldsymbol…

We study some of the most commonly used mutual information estimators, based on histograms of fixed or adaptive bin size, $k$-nearest neighbors and kernels, and focus on optimal selection of their free parameters. We examine the consistency…

In the setting where information cannot be verified, we propose a simple yet powerful information theoretical framework---the Mutual Information Paradigm---for information elicitation mechanisms. Our framework pays every agent a measure of…

Although it is widely accepted that classical information cannot travel faster than the speed of light in vacuum, the behavior of quantum correlations and quantum information propagating through actively-pumped fast-light media has not been…

Information must take up space, must weigh, and its flux must be limited. Quantum limits on communication and information storage leading to these conclusions are here described. Quantum channel capacity theory is reviewed for both steady…

We provide finite-sample distribution approximations, that are uniform in the parameter, for inference in linear mixed models. Focus is on variances and covariances of random effects in cases where existing theory fails because their…

A central object in optimal stopping theory is the single-choice prophet inequality for independent, identically distributed random variables: Given a sequence of random variables $X_1,\dots,X_n$ drawn independently from a distribution $F$,…

Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…

For a certain moment, the information volume represented in a probability space can be accurately measured by Shannon entropy. But in real life, the results of things usually change over time, and the prediction of the information volume…

Information theory tells us that if the rate of sending information across a noisy channel were above the capacity of that channel, then the transmission would necessarily be unreliable. For classical information sent over classical or…

We derive information-theoretic converses (i.e., lower bounds) for the minimum time required by any algorithm for distributed function computation over a network of point-to-point channels with finite capacity, where each node of the…

A distributed average consensus algorithm robust to a wide range of impulsive channel noise distributions is proposed. This work is the first of its kind in the literature to propose a consensus algorithm which relaxes the requirement of…

This paper proposes a unifying variational approach for proving and extending some fundamental information theoretic inequalities. Fundamental information theory results such as maximization of differential entropy, minimization of Fisher…