Related papers: The many faces of multivariate information
What does the informational complexity of dynamical networked systems tell us about intrinsic mechanisms and functions of these complex systems? Recent complexity measures such as integrated information have sought to operationalize this…
We explore a few common models on how correlations affect information. The main model considered is the Shannon mutual information $I(S:R_1,\cdots, R_i)$ over distributions with marginals $P_{S,R_i}$ fixed for each $i$, with the analogy in…
The application of the Shannon entropy to study the relationship between information and structures has yielded insights into molecular and material systems. However, the difficulty in directly observing and manipulating atoms and molecules…
The interactions between individuals play a pivotal role in shaping the structure and dynamics of social systems. Complex network models have proven invaluable in uncovering the underlying mechanisms that govern the formation and evolution…
The information shared among observables representing processes of interest is traditionally evaluated in terms of macroscale measures characterizing aggregate properties of the underlying processes and their interactions. Traditional…
Reconstructing the structural connectivity between interacting units from observed activity is a challenge across many different disciplines. The fundamental first step is to establish whether or to what extent the interactions between the…
The maximal information coefficient (MIC) is a tool for finding the strongest pairwise relationships in a data set with many variables (Reshef et al., 2011). MIC is useful because it gives similar scores to equally noisy relationships of…
There has recently been an explosion of interest in how "higher-order" structures emerge in complex systems. This "emergent" organization has been found in a variety of natural and artificial systems, although at present the field lacks a…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
Holistic analysis of many real-world problems are based on data collected from multiple sources contributing to some aspect of that problem. The word fusion has also been used in the literature for such problems involving disparate data…
We propose a framework to analyze how multivariate representations disentangle ground-truth generative factors. A quantitative analysis of disentanglement has been based on metrics designed to compare how one variable explains each…
This thesis details a class of partial orders on the space of probability distributions and the space of density operators which capture the idea of information content. Some links to domain theory and computational linguistics are also…
We investigate the information distribution among different entities in the weak measurements protocol. Focusing on multilevel, decaying systems under continuous (no-click) monitoring, we derive exact, conservation-type information…
The mutual and tripartite information between pairs and triples of disjoint regions in a quantum field theory are sensitive probes of the spread of correlations in an equilibrating system. We compute these quantities in strongly coupled…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
Multi-scale structures are prevalent in both natural and artificial systems, as they can handle increasing complexity. Several terms are employed almost interchangeably across various application domains to refer to the multi-scale concept…
Widespread use of the Internet and social networks invokes the generation of big data, which is proving to be useful in a number of applications. To deal with explosively growing amounts of data, data analytics has emerged as a critical…
The network representation is becoming increasingly popular for the description of cardiovascular interactions based on the analysis of multiple simultaneously collected variables. However, the traditional methods to assess network links…
Partial orders have been used to model several experimental setups, going from classical thermodynamics and general relativity to the quantum realm with its resource theories. In order to study such experimental setups, one typically…