Related papers: Neural Information Causality
Information causality (IC) was one of the first principles that have been invoked to bound the set of quantum correlations. For some families of correlations, this principle recovers exactly the boundary of the quantum set; for others,…
We reformulate the information causality in a more general framework by adopting the results of signal propagation and computation in a noisy circuit. In our framework, the information causality leads to a broad class of Tsirelson…
This paper develops a unified information-theoretic framework for artificial-intelligence (AI)-aided integrated sensing and communication (ISAC), where a learning component with limited representational capacity is embedded within the…
We consider the problem of reliable communication over multiple-access channels (MAC) where the channel is driven by an independent and identically distributed state process and the encoders and the decoder are provided with various degrees…
The capacity of a class of Interference Relay Channels (IRC) -the Injective Semideterministic IRC where the relay can only observe one of the sources- is investigated. We first derive a novel outer bound and two inner bounds which are based…
Information Causality is a physical principle which states that the amount of randomly accessible data over a classical communication channel cannot exceed its capacity, even if the sender and the receiver have access to a source of…
Sensory representation is typically understood through a hierarchical-causal framework where progressively abstract features are extracted sequentially. However, this causal view fails to explain misrepresentation, a phenomenon better…
The concept of causal abstraction got recently popularised to demystify the opaque decision-making processes of machine learning models; in short, a neural network can be abstracted as a higher-level algorithm if there exists a function…
A grand challenge in representation learning is to learn the different explanatory factors of variation behind the high dimen- sional data. Encoder models are often determined to optimize performance on training data when the real objective…
Identifying the physical grounds distinguishing quantum theory from broader probabilistic frameworks remains an open challenge. Communication-based proposals -- most notably the principles of impossibility of superluminal signaling and…
Representations pervade our daily experience, from letters representing sounds to bit strings encoding digital files. While such representations require externally defined decoders to convey meaning, conscious experience appears…
The neural basis of probabilistic computations remains elusive, even amidst growing evidence that humans and other animals track their uncertainty. Recent work has proposed that probabilistic representations arise naturally in…
We investigate the capacity region of multi-user interference channels (IC), where each user encodes multiple sub-user components. By unifying chain-rule decomposition with the Entropy Power Inequality (EPI), we reason that single-user…
Bell nonlocality is one of the most intriguing and counter-intuitive phenomena displayed by quantum systems. Interestingly, such stronger-than-classical quantum correlations are somehow constrained, and one important question to the…
Bayesian Networks may be appealing for clinical decision-making due to their inclusion of causal knowledge, but their practical adoption remains limited as a result of their inability to deal with unstructured data. While neural networks do…
Identifying the causal relations between interested variables plays a pivotal role in representation learning as it provides deep insights into the dataset. Identifiability, as the central theme of this approach, normally hinges on…
Information causality was proposed as a physical principle to put upper bound on the accessible information gain in a physical bi-partite communication scheme. Intuitively, the information gain cannot be larger than the amount of classical…
We investigate how to exploit intermittent feedback for interference management by studying the two-user Gaussian interference channel (IC). We approximately characterize (within a universal constant) the capacity region for the Gaussian IC…
Artificial and biological agents cannon learn given completely random and unstructured data. The structure of data is encoded in the metric relationships between data points. In the context of neural networks, neuronal activity within a…
Drawbacks of ignoring the causal mechanisms when performing imitation learning have recently been acknowledged. Several approaches both to assess the feasibility of imitation and to circumvent causal confounding and causal misspecifications…