相关论文: Remarks on the Most Informative Function Conjectur…

In this paper we focus on the estimation of mutual information from finite samples $(\mathcal{X}\times\mathcal{Y})$. The main concern with estimations of mutual information is their robustness under the class of transformations for which it…

Variational inference with a factorized Gaussian posterior estimate is a widely used approach for learning parameters and hidden variables. Empirically, a regularizing effect can be observed that is poorly understood. In this work, we show…

Motivated by data-rich experiments in transcriptional regulation and sensory neuroscience, we consider the following general problem in statistical inference. When exposed to a high-dimensional signal S, a system of interest computes a…

We propose a novel estimator of the mutual information between two ordinal vectors $x$ and $y$. Our approach is inductive (as opposed to deductive) in that it depends on the data generating distribution solely through some nonparametric…

Lower bounds for the average probability of error of estimating a hidden variable X given an observation of a correlated random variable Y, and Fano's inequality in particular, play a central role in information theory. In this paper, we…

Suppose that $Y^n$ is obtained by observing a uniform Bernoulli random vector $X^n$ through a binary symmetric channel with crossover probability $\alpha$. The "most informative Boolean function" conjecture postulates that the maximal…

Reshef et al. recently proposed a new statistical measure, the "maximal information coefficient" (MIC), for quantifying arbitrary dependencies between pairs of stochastic quantities. MIC is based on mutual information, a fundamental…

Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables. However,…

We show that the mutual information between two symbols, as a function of the number of symbols between the two, decays exponentially in any probabilistic regular grammar, but can decay like a power law for a context-free grammar. This…

We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…

While the quantum mutual information is a fundamental measure of quantum information, it is only defined for spacelike-separated quantum systems. Such a limitation is not present in the theory of classical information, where the mutual…

Factorizing low-rank matrices has many applications in machine learning and statistics. For probabilistic models in the Bayes optimal setting, a general expression for the mutual information has been proposed using heuristic statistical…

Let $\lambda (n)$ denote the Liouville function. Complementary to the prime number theorem, Chowla conjectured that \vspace{1mm} \noindent {\bf Conjecture (Chowla).} {\em \begin{equation} \label{a.1} \sum_{n\le x} \lambda (f(n)) =o(x)…

A distributed consensus algorithm for estimating the maximum value of the initial measurements in a sensor network with communication noise is proposed. In the absence of communication noise, max estimation can be done by updating the state…

This paper presents a heuristic framework for analyzing the Goldbach Conjecture (GC) from the perspective of the physics of information. Through empirical analysis, we propose an Informational Economy Principle (IEP), which posits that…

Confirmation bias, the tendency to interpret information in a way that aligns with one's preconceptions, can profoundly impact scientific research, leading to conclusions that reflect the researcher's hypotheses even when the observational…

We show that there are informationally complete joint measurements of two conjugated observables on a finite quantum system, meaning that they enable to identify all quantum states from their measurement outcome statistics. We further…

This paper deals with the parametric inference for integrated signals embedded in an additive Gaussian noise and observed at deterministic discrete instants which are not necessarily equidistant. The unknown parameter is multidimensional…

The quadratic decaying property of the information rate function states that given a fixed conditional distribution $p_{\mathsf{Y}|\mathsf{X}}$, the mutual information between the (finite) discrete random variables $\mathsf{X}$ and…

I consider the tradeoff between the information gained about an initially unknown quantum state, and the disturbance caused to that state by the measurement process. I show that for any distribution of initial states, the…