相关论文: Common Information, Noise Stability, and Their Ext…
Let $ X_{n} $ be $ n\times N $ random complex matrices, $R_{n}$ and $T_{n}$ be non-random complex matrices with dimensions $n\times N$ and $n\times n$, respectively. We assume that the entries of $ X_{n} $ are independent and identically…
Background: Alignment of biological sequences such as DNA, RNA or proteins is one of the most widely used tools in computational bioscience. All existing alignment algorithms rely on heuristic scoring schemes based on biological expertise.…
In a previous report we have evaluated analytically the mutual information between the firing rates of N independent units and a set of multi-dimensional continuous+discrete stimuli, for a finite population size and in the limit of large…
The concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) are introduced to measure the concentration and separation performance of a classification deep neural network (DNN) in the output…
Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this…
Understanding the source of the superior generalization ability of NNs remains one of the most important problems in ML research. There have been a series of theoretical works trying to derive non-vacuous bounds for NNs. Recently, the…
We establish the first known upper bound on the exact and Wyner's common information of $n$ continuous random variables in terms of the dual total correlation between them (which is a generalization of mutual information). In particular, we…
This paper investigates a multi-terminal source coding problem under a logarithmic loss fidelity which does not necessarily lead to an additive distortion measure. The problem is motivated by an extension of the Information Bottleneck…
Integrated Information Theory (IIT) is an audacious attempt to pin down the abstract, phenomenological experiences of consciousness into a rigorous, mathematical framework. We show that IIT's stance in regards to neuronal noise is…
In this paper, we revisit the discrete lossy Gray-Wyner problem. In particular, we derive its optimal second-order coding rate region, its error exponent (reliability function) and its moderate deviations constant under mild conditions on…
Quantum information theory is the study of the achievable limits of information processing within quantum mechanics. Many different types of information can be accommodated within quantum mechanics, including classical information, coherent…
Recent work by Bravyi et al. constructs a relation problem that a noisy constant-depth quantum circuit (QNC$^0$) can solve with near certainty (probability $1 - o(1)$), but that any bounded fan-in constant-depth classical circuit (NC$^0$)…
Learning good representations is of crucial importance in deep learning. Mutual Information (MI) or similar measures of statistical dependence are promising tools for learning these representations in an unsupervised way. Even though the…
We compared the noise correlation and the success probability of coherent Ising machines (CIMs) with optical delay-line, measurement feedback, and mean-field couplings. We theoretically studied three metrics for the noise correlations in…
The generalized amplitude damping channel (GADC) is one of the sources of noise in superconducting-circuit-based quantum computing. It can be viewed as the qubit analogue of the bosonic thermal channel, and it thus can be used to model…
In social networks, information and influence diffuse among users as cascades. While the importance of studying cascades has been recognized in various applications, it is difficult to observe the complete structure of cascades in practice.…
We introduce and study the cumulative information generating function, which provides a unifying mathematical tool suitable to deal with classical and fractional entropies based on the cumulative distribution function and on the survival…
This paper studies the two-user Causal Cognitive Interference Channel (CCIC), where two transmitters aim to communicate independent messages to two different receivers via a common channel. One source, referred to as the cognitive, is…
In array processing, a common problem is to estimate the angles of arrival of $K$ deterministic sources impinging on an array of $M$ antennas, from $N$ observations of the source signal, corrupted by gaussian noise. The problem reduces to…
We introduce a new information-theoretic complexity measure $IC_\infty$ for 2-party functions which is a lower-bound on communication complexity, and has the two leading lower-bounds on communication complexity as its natural relaxations:…