Related papers: A new approach to mutual information. II
This article is an introduction to newly discovered relations between volumes of moduli spaces of Riemann surfaces or super Riemann surfaces, simple models of gravity or supergravity in two dimensions, and random matrix ensembles. (The…
We prove that mutual information is actually negative copula entropy, based on which a method for mutual information estimation is proposed.
A new combinatorial-probabilistic diagnostic entropy has been introduced. It describes the pair-wise sum of probabilities of system conditions that have to be distinguished during the diagnosing process. The proposed measure describes the…
We consider the problem of decomposing the total mutual information conveyed by a pair of predictor random variables about a target random variable into redundant, unique and synergistic contributions. We focus on the relationship between…
We define Persistent Mutual Information (PMI) as the Mutual (Shannon) Information between the past history of a system and its evolution significantly later in the future. This quantifies how much past observations enable long term…
A new approach is presented to compute entropy for massless scalar quantum fields. By perturbing a skewed correlation matrix composed of field operator correlation functions, the mutual information is obtained for disjoint spherical regions…
Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…
This correspondence studies the basic problem of classifications - how to evaluate different classifiers. Although the conventional performance indexes, such as accuracy, are commonly used in classifier selection or evaluation,…
We derive general upper bounds to pointwise mutual information in terms of stochastic Fisher information and show these bounds average to known results in the literature for bounds to mutual information in terms of Fisher information. These…
To study the communication between information systems, Wang et al. [C. Wang, C. Wu, D. Chen, Q. Hu, and C. Wu, Communicating between information systems, Information Sciences 178 (2008) 3228-3239] proposed two concepts of type-1 and type-2…
We propose new measures of shared information, unique information and synergistic information that can be used to decompose the multi-information of a pair of random variables $(Y,Z)$ with a third random variable $X$. Our measures are…
The mutual information is a measure of classical and quantum correlations of great interest in quantum information. It is also relevant in quantum many-body physics, by virtue of satisfying an area law for thermal states and bounding all…
Recently, Mutual Information (MI) has attracted attention in bounding the generalization error of Deep Neural Networks (DNNs). However, it is intractable to accurately estimate the MI in DNNs, thus most previous works have to relax the MI…
We show that the mutual information between two symbols, as a function of the number of symbols between the two, decays exponentially in any probabilistic regular grammar, but can decay like a power law for a context-free grammar. This…
The ability of information processing in biologically motivated Boolean networks is of interest in recent information theoretic research. One measure to quantify this ability is the well known mutual information. Using Fourier analysis we…
We introduce and investigate in this paper a new convenient method of introduction of a norm in the multidimensional rearrangement probability invariant space.
The weighted median mechanism provides a robust alternative to weighted averaging in opinion dynamics. Existing models, however, are predominantly formulated on pairwise interaction graphs, which limits their ability to represent…
Information thermodynamics relates the rate of change of mutual information between two interacting subsystems to their thermodynamics when the joined system is described by a bipartite stochastic dynamics satisfying local detailed balance.…
In this article, we extend the foundations of the theory of differential inclusions in the space of compactly supported probability measures, introduced recently by the authors, to the setting of general Wasserstein spaces. In this context,…
We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections, a problem relevant in compressed sensing, sparse superposition codes or code division multiple access just to cite few. There has…