Related papers: On essentially conditional information inequalitie…
Determining the strength of non-linear statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual…
Deriving formulations for computing and estimating tight worst-case size increases for conjunctive queries with various constraints has been at the core of theoretical database research. If the problem has no constraints or only one…
We prove an inequality for the entropy numbers in terms of nonlinear Kolmogorov's widths. This inequality is in a spirit of known inequalities of this type and it is adjusted to the form convenient in applications for $m$-term…
The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. In this paper, we establish a number of simple inequalities for the weighted entropies…
The classical causal relations between a set of variables, some observed and some latent, can induce both equality constraints (typically conditional independences) as well as inequality constraints (Instrumental and Bell inequalities being…
It is well known that there is a strong connection between entropy inequalities and submodularity, since the entropy of a collection of random variables is a submodular function. Unifying frameworks for information inequalities arising from…
We show that there is equality in Shannon's Entropy Power Inequality (EPI) if and only if the random variables involved are Gaussian, assuming nothing beyond the existence of differential entropies. This is done by justifying de Bruijn's…
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of…
The existence of observables that are incompatible or not jointly measurable is a characteristic feature of quantum mechanics, which lies at the root of a number of nonclassical phenomena, such as uncertainty relations, wave--particle dual…
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
In most communication schemes information is transmitted via travelling modes of electromagnetic radiation. These modes are unavoidably subject to environmental noise along any physical transmission medium and the quality of the…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
The thermodynamic uncertainty relation is an inequality stating that it is impossible to attain higher precision than the bound defined by entropy production. In statistical inference theory, information inequalities assert that it is…
Fano's inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano's inequality is generalized to a broad class of information measures, which contains those of Shannon…
Information theory has provided foundations for the theories of several application areas critical for modern society, including communications, computer storage, and AI. A key aspect of Shannon's 1948 theory is a sharp lower bound on the…
We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon's information theory that takes into account the modeling power and computational constraints of…
Given an independent and identically distributed source $X = \{X_i \}_{i=1}^{\infty}$ with finite Shannon entropy or differential entropy (as the case may be) $H(X)$, the non-asymptotic equipartition property (NEP) with respect to $H(X)$ is…
Recently it was shown that if a given state fulfils the reduction criterion it must also satisfy the known entropic inequalities. Now the questions arises whether on the assumption that stronger criteria based on positive but not completely…
Classical and quantum physics impose different constraints on the joint probability distributions of observed variables in a causal structure. These differences mean that certain correlations can be certified as non-classical, which has…
We introduce the (private) entropy of a directed graph (in a new network coding sense) as well as a number of related concepts. We show that the entropy of a directed graph is identical to its guessing number and can be bounded from below…