Related papers: A Logarithmic Decomposition for Information
Mutual information $I(X;Y)$ is a useful definition in information theory to estimate how much information the random variable $Y$ holds about the random variable $X$. One way to define the mutual information is by comparing the joint…
In this paper we define and study signed deficient topological measures and signed topological measures (which generalize signed measures) on locally compact spaces. We prove that a signed deficient topological measure is $\tau$-smooth on…
The Partial Information Decomposition (PID) takes one step beyond Shannon's theory in decomposing the information two variables $A,B$ possess about a third variable $T$ into distinct parts: unique, shared (or redundant) and synergistic…
We provide a stochastic extension of the Baez-Fritz-Leinster characterization of the Shannon information loss associated with a measure-preserving function. This recovers the conditional entropy and a closely related information-theoretic…
It is known that the high-dimensional quantum state space is notoriously complicated in contrast with the beautiful Bloch ball of the qubit. We examined the mechanism behind this fact in the frame work of general probabilistic theory (GPT),…
Information theory has provided foundations for the theories of several application areas critical for modern society, including communications, computer storage, and AI. A key aspect of Shannon's 1948 theory is a sharp lower bound on the…
The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector of d factors having a density w.r.t. the Lebesgue measure in R^d. Namely, the…
Given a probability measure $\mu$ over ${\mathbb R}^n$, it is often useful to approximate it by the convex combination of a small number of probability measures, such that each component is close to a product measure. Recently, Ronen Eldan…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
According to E.T. Jaynes and E.P. Wigner, entropy is an anthropomorphic concept in the sense that in a physical system correspond many thermodynamic systems. The physical system can be examined from many points of view each time examining…
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label…
We show that there is a unique maximal decomposition of a pure multi-partite (N>2) quantum state into a sum of states which are "locally orthogonal" in the sense that the local reduced state for a term in the sum lives in its own orthogonal…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
After Shannon, entropy becomes a fundamental quantity to describe not only uncertainity or chaos of a system but also information carried by the system. Shannon's important discovery is to give a mathematical expression of the mutual…
The Shannon based conditional entropy that underlies five-dimensional Einstein-Hilbert gravity coupled to a dilaton field is investigated in the context of dynamical holographic AdS/QCD models. Considering the UV and IR dominance limits of…
The scaling properties of various composite information-theoretic measures (Shannon and R\'enyi entropy sums, Fisher and Onicescu information products, Tsallis entropy ratio, Fisher-Shannon product and shape complexity) are studied in…
A unified combinatorial definition of the information content and entropy of different types of patterns, compatible with the traditional concepts of information and entropy, going beyond the limitations of Shannon information interpretable…
A direct connection of information entropy $S$ and kinetic energy $T$ is obtained for nuclei and atomic clusters, which establishes $T$ as a measure of the information in a distribution. It is conjectured that this is a universal property…
We investigate the quantum information by a theoretical measurement approach of an Aharanov-Bohm (AB) ring with Yukawa interaction in curved space with disclination. It obtained the so-called Shannon entropy, through the eigenfunctions of…
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…