Related papers: Orders between channels and implications for parti…
The interest in channel models in which the data is sent as an unordered set of binary strings has increased lately, due to emerging applications in DNA storage, among others. In this paper we analyze the minimal redundancy of binary codes…
Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of redundancy lattices, which allows separating the information that a set of variables contains about a target variable into…
We consider an approximation scheme for multivariate information assuming that synergistic information only appearing in higher order joint distributions is suppressed, which may hold in large classes of systems. Our approximation scheme…
The data processing inequality is the most basic requirement for any meaningful measure of information. It essentially states that distinguishability measures between states decrease if we apply a quantum channel and is the centerpiece of…
An important issue during an engineering design process is to develop an understanding which design parameters have the most influence on the performance. Especially in the context of optimization approaches this knowledge is crucial in…
Quantifying modality contributions in multimodal models remains a challenge, as existing approaches conflate the notion of contribution itself. Prior work relies on accuracy-based approaches, interpreting performance drops after removing a…
A polar decomposition of mutual information between a complex-valued channel's input and output is proposed for a input whose amplitude and phase are independent of each other. The mutual information is symmetrically decomposed into three…
A fundamental question in information theory is to quantify the loss of information under a noisy channel. Partial orders and contraction coefficients are typical tools to that end, however, they are often also challenging to evaluate. For…
This thesis details a class of partial orders on the space of probability distributions and the space of density operators which capture the idea of information content. Some links to domain theory and computational linguistics are also…
The decomposition of channel information into synergies of different order is an open, active problem in the theory of complex systems. Most approaches to the problem are based on information theory, and propose decompositions of mutual…
A core feature of complex systems is that the interactions between elements in the present causally constrain each-other as the system evolves through time. To fully model all of these interactions (between elements, as well as ensembles of…
In this paper we present an unconventional image segmentation approach which is devised to meet the requirements of image understanding and pattern recognition tasks. Generally image understanding assumes interplay of two sub-processes:…
Integrated information theory is a mathematical, quantifiable theory of conscious experience. The linchpin of this theory, the $\phi$ measure, quantifies a system's irreducibility to disjoint parts. Purely as a measure of irreducibility, we…
Multimodal learning significantly benefits cancer survival prediction, especially the integration of pathological images and genomic data. Despite advantages of multimodal learning for cancer survival prediction, massive redundancy in…
The entropy of a pair of random variables is commonly depicted using a Venn diagram. This representation is potentially misleading, however, since the multivariate mutual information can be negative. This paper presents new measures of…
Mutual information between two random variables is a well-studied notion, whose understanding is fairly complete. Mutual information between one random variable and a pair of other random variables, however, is a far more involved notion.…
The properties of complex networked systems arise from the interplay between the dynamics of their elements and the underlying topology. Thus, to understand their behaviour, it is crucial to convene as much information as possible about…
A key step in reverse engineering neural networks is to decompose them into simpler parts that can be studied in relative isolation. Linear parameter decomposition -- a framework that has been proposed to resolve several issues with current…
In previous work, an ordering result was given for the symbolwise probability of error using general Markov channels, under iterative decoding of LDPC codes. In this paper, the ordering result is extended to mutual information, under the…
Information decompositions quantify how the Shannon information about a given random variable is distributed among several other random variables. Various requirements have been proposed that such a decomposition should satisfy, leading to…