Related papers: Inequalities Revisited
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
The entropy region is a fundamental object in information theory. An outer bound for the entropy region is defined by a minimal set of Shannon-type inequalities called elemental inequalities also referred to as the Shannon region. This…
Uncertainty relations based on information theory for both discrete and continuous distribution functions are briefly reviewed. We extend these results to account for (differential) R\'{e}nyi entropy and its related entropy power. This…
Information-theoretic inequalities play a fundamental role in numerous scientific and technological areas as they generally express the impossibility to have a complete description of a system via a finite number of information measures. In…
The article builds on several recent advances in the Monge-Kantorovich theory of mass transport which have -- among other things -- led to new and quite natural proofs for a wide range of geometric inequalities such as the ones formulated…
In this article we discuss the formal structure of a generalized information theory based on the extension of the probability calculus of Kolmogorov to a (possibly) non-commutative setting. By studying this framework, we argue that quantum…
The aim of the paper is to study the link between non additivity of some entropies and their boundedness. We propose an axiomatic construction of the entropy relying on the fact that entropy belongs to a group isomorphic to the usual…
Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning. While numerous…
Shannon Information theory has achieved great success in not only communication technology where it was originally developed for but also many other science and engineering fields such as machine learning and artificial intelligence.…
Algorithmic entropy and Shannon entropy are two conceptually different information measures, as the former is based on size of programs and the later in probability distributions. However, it is known that, for any recursive probability…
Entropy is a fundamental property of both classical and quantum systems, spanning myriad theoretical and practical applications in physics and computer science. We study the problem of obtaining estimates to within a multiplicative factor…
Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…
In this paper, we first provide three general norm inequalities, which are used to give new uncertainty relations of any finite observables and quantum channels via metric-adjusted skew information. The results are applicable to its special…
In 1998, Zhang and Yeung found the first unconditional non-Shannon-type information inequality. Recently, Dougherty, Freiling and Zeger gave six new unconditional non-Shannon-type information inequalities. This work generalizes their work…
The information-theoretic approach to Bell's theorem is developed with use of the conditional $q$-entropies. The $q$-entropic measures fulfill many similar properties to the standard Shannon entropy. In general, both the locality and…
We investigate the universal inequalities relating the alpha-Renyi entropies of the marginals of a multi-partite quantum state. This is in analogy to the same question for the Shannon and von Neumann entropy (alpha=1) which are known to…
We consider the Shannon-Khinchin axiomatic systems for the characterization of generalized entropies such as Sharma-Mital and Frank-Daffertshofer entropy. We provide the generalization of Shannon-Khinchin axioms and give the corresponding…
The Clebsch-Gordan coefficients of the group SU(2) are shown to satisfy new inequalities. They are obtained using the properties of Shannon and Tsallis entropies. The inequalities associated with the Wigner 3-j symbols are obtained using…
The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…