Related papers: The Common Information of N Dependent Random Varia…
As conventional communication systems based on classic information theory have closely approached the limits of Shannon channel capacity, semantic communication has been recognized as a key enabling technology for the further improvement of…
We discuss the Dempster-Shafer theory of evidence. We introduce a concept of monotonicity which is related to the diminution of the range between belief and plausibility. We show that the accumulation of knowledge in this framework exhibits…
This paper investigates Distributed Hypothesis testing (DHT), in which a source $\mathbf{X}$ is encoded given that side information $\mathbf{Y}$ is available at the decoder only. Based on the received coded data, the receiver aims to decide…
We prove rigorously a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method. For general random variables $(Y, X)$, we show…
We propose a method for inferring the existence of a latent common cause ('confounder') of two observed random variables. The method assumes that the two effects of the confounder are (possibly nonlinear) functions of the confounder plus…
Both sampling a time-varying signal, and its spectral analysis are activities subjected to theoretically compelling, such as Shannon's theorem and the objectively limiting of the frequency's resolution. Usually, the signals' spectra are…
A short, information-theoretic proof of the Kac--Bernstein theorem, which is stated as follows, is presented: For any independent random variables $X$ and $Y$, if $X+Y$ and $X-Y$ are independent, then $X$ and $Y$ are normally distributed.
A family of skew information quantities is obtained, in which the well-known Wigner-Yanase skew information and quantum Fisher information stand as special cases. A transparent proof of convexity of the generalized skew information is…
Fano's inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano's inequality is generalized to a broad class of information measures, which contains those of Shannon…
The conditional quantum mutual information $I(A;B|C)$ of a tripartite state $\rho_{ABC}$ is an information quantity which lies at the center of many problems in quantum information theory. Three of its main properties are that it is…
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits…
We generalize Lindeberg's proof of the central limit theorem to an invariance principle for arbitrary smooth functions of independent and weakly dependent random variables. The result is applied to get a similar theorem for smooth functions…
This paper is devoted to the mathematical study of some divergences based on the mutual information well-suited to categorical random vectors. These divergences are generalizations of the "entropy distance" and "information distance". Their…
In computer science, we can theoretically neatly separate transmission and processing of information, hardware and software, and programs and their inputs. This is much more intricate in biology, Nevertheless, I argue that Shannon's concept…
Researchers have developed ways to generalize the mean and variance to situations in which a data metric is available. We apply the tools developed in Pennec (2006) to categorical data, and show the generality of this approach by…
Bivariate Partial Information Decomposition (PID) describes how the mutual information between a random variable M and two random variables Y and Z is decomposed into unique, redundant, and synergistic terms. Recently, PID has shown promise…
The possibility to recover the which-way information, for example in the two slit experiment, is based on a natural but implicit assumption about the position of a particle {\it before} a position measurement is performed on it. This…
Any measurement is intended to provide information on a system, namely knowledge about its state. However, we learn from quantum theory that it is generally impossible to extract information without disturbing the state of the system or its…
This paper addresses the problem of inferring circulation of information between multiple stochastic processes. We discuss two possible frameworks in which the problem can be studied: directed information theory and Granger causality. The…
In general, if there is one device A with the same performance as many devices B, it would be better to replace many devices with one device. In order to determine the number of devices that can be reduced, it is important to determine the…