Related papers: The Common Information of N Dependent Random Varia…
Conceptually, partial information decomposition (PID) is concerned with separating the information contributions several sources hold about a certain target by decomposing the corresponding joint mutual information into contributions such…
In this paper, we prove the existence of fundamental relations between information theory and estimation theory for network-coded flows. When the network is represented by a directed graph G=(V, E) and under the assumption of uncorrelated…
We consider a setting of Slepian--Wolf coding, where the random bin of the source vector undergoes channel coding, and then decoded at the receiver, based on additional side information, correlated to the source. For a given distribution of…
We study a separable design for computing information measures, where the information measure is computed from learned feature representations instead of raw data. Under mild assumptions on the feature representations, we demonstrate that a…
The problem of side-information scalable (SI-scalable) source coding is considered in this work, where the encoder constructs a progressive description, such that the receiver with high quality side information will be able to truncate the…
We address three outstanding problems in information theory. Problem one is the definition of a non-negative decomposition of the information conveyed by two or more sources about a target variable into the specific contribution of each…
Information theory is introduced in this lecture note with a particular emphasis on its relevance to algebraic coding theory. The document develops the mathematical foundations for quantifying uncertainty and information transmission by…
Wiener and Granger have introduced an intuitive concept of causality between two variables which is based on the idea that an effect never occurs before its cause. Later, Geweke has generalized this concept to a multivariate Granger…
Quantum information is a common topic of research in many areas of quantum physics, such as quantum communication and quantum computation, as well as quantum thermodynamics. It can be encoded in discrete or continuous variable systems, with…
We establish a precise mathematical equivalence between witness-based similarity systems (REWA) and Shannon's information theory. We prove that witness overlap is mutual information, that REWA bit complexity bounds arise from channel…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
In this paper we consider the axiomatic characterization of information and certainty measures in a unified way. We present the general axiomatic system which captures the common properties of a large number of the measures previously…
In this note, a Wegner estimate for random divergence-type operators that are monotone in the randomness is proven. The proof is based on a recently shown unique continuation estimate for the gradient and the ensuing eigenvalue liftings.…
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios.…
In this work, we propose two information generating functions: general weighted information and relative information generating functions, and study their properties. { It is shown that the general weighted information generating function…
As was noted already by A. N. Kolmogorov, any random variable has a Bernoulli component. This observation provides a tool for the extension of results which are known for Bernoulli random variables to arbitrary distributions. Two…
Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic…
General characterization of physical measurements is discussed within the framework of a classical information theory. Uncertainty relation for simultaneous measurements of two physical observables is defined in this framework for…
The network communication scenario where one or more receivers request all the information transmitted by different sources is considered. We introduce distributed polynomial-time network codes in the presence of malicious nodes. Our codes…
Confounding seriously impairs our ability to learn about causal relations from observational data. Confounding can be defined as a statistical association between two variables due to inputs from a common source (the confounder). For…