Related papers: Structural Impossibility of Antichain-Lattice Part…
Systems of interest for theoretical or experimental work often exhibit high-order interactions, corresponding to statistical interdependencies in groups of variables that cannot be reduced to dependencies in subsets of them. While still…
This paper studies the Pliable Index CODing problem (PICOD), which models content-type distribution networks. In the PICOD$(t)$ problem there are $m$ messages, $n$ users and each user has a distinct message side information set, as in the…
Integrated information theory (IIT) starts from consciousness itself and identifies a set of properties (axioms) that are true of every conceivable experience. The axioms are translated into a set of postulates about the substrate of…
Resistive memories store information in a crossbar arrangement of two-terminal devices that can be programmed to patterns of high or low resistance. While extremely compact, this technology suffers from the "sneak-path" problem: certain…
We propose an effect called information constraint which is characterized by the existence of different decay rates of signal strengths propagating along opposite directions. It is an intrinsic property of a type of open quantum system,…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
When a link is occupied to restrict the growth of large clusters using the size information of a finite number of finite clusters, so-called local information, an abrupt but continuous transition is exhibited. We report here that a hybrid…
We propose an information-theoretic private information retrieval (PIR) scheme for distributed storage systems where data is stored using a linear systematic code of rate $R > 1/2$. The proposed scheme generalizes the PIR scheme for data…
Recent contrastive representation learning methods rely on estimating mutual information (MI) between multiple views of an underlying context. E.g., we can derive multiple views of a given image by applying data augmentation, or we can…
The key challenge in unaligned multimodal language sequences lies in effectively integrating information from various modalities to obtain a refined multimodal joint representation. Recently, the disentangle and fuse methods have achieved…
As interpretability gains attention in machine learning, there is a growing need for reliable models that fully explain representation content. We propose a mutual information (MI)-based method that decomposes neural network representations…
We build information geometry for a partially ordered set of variables and define the orthogonal decomposition of information theoretic quantities. The natural connection between information geometry and order theory leads to efficient…
A core feature of complex systems is that the interactions between elements in the present causally constrain each-other as the system evolves through time. To fully model all of these interactions (between elements, as well as ensembles of…
We consider partially-specified optimization problems where the goal is to actively, but efficiently, acquire missing information about the problem in order to solve it. An algorithm designer wishes to solve a linear program (LP), $\max…
Quantifying integrated information is a leading approach towards building a fundamental theory of consciousness. Integrated Information Theory (IIT) has gained attention in this regard due to its theoretically strong framework. However, it…
We present a theoretical framework that extends classical information theory to finite and structured systems by redefining redundancy as a fundamental property of information organization rather than inefficiency. In this framework,…
Intrinsic image decomposition (IID) is the task that decomposes a natural image into albedo and shade. While IID is typically solved through supervised learning methods, it is not ideal due to the difficulty in observing ground truth albedo…
We prove the conjectured existence of Bound Information, a classical analog of bound entanglement, in the multipartite scenario. We give examples of tripartite probability distributions from which it is impossible to extract any kind of…
Quantum logical axiomatic systems for quantum theory usually include a postulate that a lattice under consideration is orthomodular. We propose a derivation of orthomodularity from an information-theoretic axiom. This provides conceptual…
This paper offers a characterization of fundamental limits on the classification and reconstruction of high-dimensional signals from low-dimensional features, in the presence of side information. We consider a scenario where a decoder has…