Related papers: Neural Information Causality
In many scenarios, human decisions are explained based on some high-level concepts. In this work, we take a step in the interpretability of neural networks by examining their internal representation or neuron's activations against concepts.…
Implicit Neural Representations (INRs) are widely used to encode data as continuous functions, enabling the visualization of large-scale multivariate scientific simulation data with reduced memory usage. However, existing INR-based methods…
Quantum resources can, in principle, enable Heisenberg-limited (HL) sensing, yet no-go theorems imply that HL scaling is generically unattainable in realistic noisy devices. While quantum error correction (QEC) can suppress noise, its use…
Due to structural and functional abnormalities or genetic variations and mutations, there may be dysfunctional molecules within an intracellular signaling network that do not allow the network to correctly regulate its output molecules,…
The problem of bistatic integrated sensing and communications over memoryless relay channels is considered, where destination concurrently decodes the message sent by the source and estimates unknown parameters from received signals with…
This paper is concerned with general interference channels characterized by a sequence of transition (conditional) probabilities. We present a general formula for the capacity region of the interference channel with two pairs of users. The…
A $n^d \xrightarrow{p} 1$ Quantum Random Access Code (QRAC) is a communication task where Alice encodes $n$ classical bits into quantum states of dimension $d$ and transmits them to Bob, who performs appropriate measurements to recover the…
Reasoning, the ability to logically draw conclusions from existing knowledge, is a hallmark of human. Together with perception, they constitute the two major themes of artificial intelligence. While deep learning has pushed the limit of…
A long standing open problem in the theory of neural networks is the development of quantitative methods to estimate and compare the capabilities of different architectures. Here we define the capacity of an architecture by the binary…
Contemporary machine learning optimizes for predictive accuracy, yet systems that achieve state of the art performance remain causally opaque: their internal representations provide no principled handle for intervention. We can retrain such…
In these series of multi-part papers, a systematic study of fundamental limits of communications in interference networks is established. Here, interference network is referred to as a general single-hop communication scenario with…
The Abstraction and Reasoning Corpus (ARC) provides a compact laboratory for studying abstract reasoning, an ability central to human intelligence. Modern AI systems, including LLMs and ViTs, largely operate as sequence-of-behavior…
Given two independent point processes and a certain rule for matching points between them, what is the fraction of matched points over infinitely long streams? In many application contexts, e.g., secure networking, a meaningful matching…
The human cognitive system exhibits remarkable flexibility and generalization capabilities, partly due to its ability to form low-dimensional, compositional representations of the environment. In contrast, standard neural network…
Faithful evaluation of language model capabilities is crucial for deriving actionable insights that can inform model development. However, rigorous causal evaluations in this domain face significant methodological challenges, including…
This paper reviews, analyzes, and proposes a new perspective on the bi-encoder architecture for neural search. While the bi-encoder architecture is widely used due to its simplicity and scalability at test time, it has some notable issues…
The quantum enhancement of success probability in the Random Access Code (RAC) protocols remains unexplored from two important perspectives. First, the use of entanglement between two co-measurable degrees of freedom of a single particle…
We study the capacity of discrete memoryless many-to-one interference channels, i.e., K user interference channels where only one receiver faces interference. For a class of many-to-one interference channels, we identify a noisy…
In this study, we design a low-complexity and generalized AI model that can capture common knowledge to improve data reconstruction of the channel decoder for semantic communication. Specifically, we propose a generative adversarial network…
Quantum reservoir computing (QRC) harnesses driven quantum dynamics for time-series processing, yet the mechanisms behind the differing performance levels across its many implementations remain unclear. We show that apparently unrelated…