Related papers: Conditional Information Inequalities for Entropic …
We exhibit infinitely many new, constrained inequalities for the von Neumann entropy, and show that they are independent of each other and the known inequalities obeyed by the von Neumann entropy (basically strong subadditivity). The new…
The general idea of information entropy provided by C.E. Shannon "hangs over everything we do" and can be applied to a great variety of problems once the connection between a distribution and the quantities of interest is found. The Shannon…
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
Based on the form invariance of the structures given by Khinchin's axiomatic foundations of information theory and the pseudoadditivity of the Tsallis entropy indexed by q, the concept of conditional entropy is generalized to the case of…
Using joint Shannon entropy, we propose an inequality for a four-level system, which is satisfied in any non-contextual realist hidden variable model. We show that this entropic inequality is violated by quantum mechanics for a range of…
We investigate quantum R\'enyi entropic quantities, specifically those derived from 'sandwiched' divergence. This divergence is one of several proposed R\'enyi generalisations of the quantum relative entropy. We may define R\'enyi…
In this paper, we characterize the saturation of four universal inequalities in quantum information theory, including a variant version of strong subadditivity inequality for von Neumann entropy, the coherent information inequality, the…
Information is a valuable asset for agents in socio-economic systems, a significant part of the information being entailed into the very network of connections between agents. The different interlinkages patterns that agents establish may,…
We study constrained versions of the Ingleton inequality in the entropic setting and quantify its stability under small violations of conditional independence. Although the classical Ingleton inequality fails for general entropy profiles,…
The quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, has recently become quite common in machine learning. While the properties of these measures, which are rooted…
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality recently proposed by Pawlowski et. al. (arXiv:0905.2992). We consider two…
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a…
In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…
In this paper we investigate a quantity called conditional entropy of ordinal patterns, akin to the permutation entropy. The conditional entropy of ordinal patterns describes the average diversity of the ordinal patterns succeeding a given…
Uncertainty relations are central to quantum physics. While they were originally formulated in terms of variances, they have later been successfully expressed with entropies following the advent of Shannon information theory. Here, we…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…
We address the problem of applying the Kolmogorov-Sinai method of entropic analysis, expressed in a generalized non-extensive form, to the dynamics of the logistic map at the chaotic threshold, which is known to be characterized by a power…
The minimum error entropy (MEE) criterion has been successfully used in fields such as parameter estimation, system identification and the supervised machine learning. There is in general no explicit expression for the optimal MEE estimate…