Related papers: Quantifying the information transmitted in a singl…
Renormalization is an essential technique in field-theoretic descriptions of natural phenomena, where the absence of a UV-complete description yields an abundance of divergent quantities. While the renormalization prescription has been…
We study entanglement-assisted quantum and classical communication over a single use of a quantum channel, which itself can correspond to a finite number of uses of a channel with arbitrarily correlated noise. We obtain characterizations of…
We define a metric, mutual information in frequency (MI-in-frequency), to detect and quantify the statistical dependence between different frequency components in the data, referred to as cross-frequency coupling and apply it to…
Given a set of several inputs into a system (e.g., independent variables characterizing stimuli) and a set of several stochastically non-independent outputs (e.g., random variables describing different aspects of responses), how can one…
This paper considers the problem of defining a measure of redundant information that quantifies how much common information two or more random variables specify about a target random variable. We discussed desired properties of such a…
$H$-mutual information ($H$-MI) is a wide class of information leakage measures, where $H=(\eta, F)$ is a pair of monotonically increasing function $\eta$ and a concave function $F$, which is a generalization of Shannon entropy. $H$-MI is…
Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
The activity of neurons within brain circuits has been ubiquitously reported to be correlated. The impact of these correlations on brain function has been extensively investigated. Correlations can in principle increase or decrease the…
Shannon's mutual information of a random multiple antenna and multipath channel is studied in the general case where the channel impulse response is an ergodic and stationary process which is assumed to be available at the receiver. From…
In statistical physics entropy is usually introduced as a global quantity which expresses the amount of information that would be needed to specify the microscopic configuration of a system. However, for lattice models with infinitely many…
I introduce an algorithm to detect one-way quantum information between two interacting quantum systems, i.e. the direction and orientation of the information transfer in arbitrary quantum dynamics. I then build an information-theoretic…
Mutual information between two random variables is a well-studied notion, whose understanding is fairly complete. Mutual information between one random variable and a pair of other random variables, however, is a far more involved notion.…
In quantum information theory it is generally accepted that quantum mutual information is an information-theoretic measure of total correlations of a bipartite quantum state. We argue that there exist quantum states for which quantum mutual…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on…
In this paper, we present a new multi-scale information content calculation method based on Shannon information (and Shannon entropy). The original method described by Claude E. Shannon and based on the logarithm of the probability of…
Basic quantum information measures involved in the information analysis of quantum systems are considered. It is shown that the main quantum information measurement methods depend on whether the corresponding quantum events are compatible…
The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that based on mutual information, for a class of structured spiking neuronal network. In…
We define Persistent Mutual Information (PMI) as the Mutual (Shannon) Information between the past history of a system and its evolution significantly later in the future. This quantifies how much past observations enable long term…