Related papers: Mutual information rate and bounds for it
The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission…
Measures of information transfer have become a popular approach to analyze interactions in complex systems such as the Earth or the human brain from measured time series. Recent work has focused on causal definitions of information transfer…
In recent years, there has been a significant increase in applications of multimodal signal processing and analysis, largely driven by the increased availability of multimodal datasets and the rapid progress in multimodal learning systems.…
The ability to train randomly initialised deep neural networks is known to depend strongly on the variance of the weight matrices and biases as well as the choice of nonlinear activation. Here we complement the existing geometric analysis…
One of the most fundamental questions one can ask about a pair of random variables X and Y is the value of their mutual information. Unfortunately, this task is often stymied by the extremely large dimension of the variables. We might hope…
Given a pair of predictor variables and a response variable, how much information do the predictors have about the response, and how is this information distributed between unique, redundant, and synergistic components? Recent work has…
Random Threshold Networks (RTNs) are an idealized model of diluted, non symmetric spin glasses, neural networks or gene regulatory networks. RTNs also serve as an interesting general example of any coordinated causal system. Here we study…
The information transmission rate (ITR), or effective bit rate, is a popular and widely used information measurement metric, particularly popularized for SSVEP-based Brain-Computer (BCI) interfaces. By combining speed and accuracy into a…
The correlation distance quantifies the statistical independence of two classical or quantum systems, via the distance from their joint state to the product of the marginal states. Tight lower bounds are given for the mutual information…
We study some of the most commonly used mutual information estimators, based on histograms of fixed or adaptive bin size, $k$-nearest neighbors and kernels, and focus on optimal selection of their free parameters. We examine the consistency…
A well-known metric for quantifying the similarity between two clusterings is the adjusted mutual information. Compared to mutual information, a corrective term based on random permutations of the labels is introduced, preventing two…
Channel capacity plays a crucial role in the development of modern communication systems as it represents the maximum rate at which information can be reliably transmitted over a communication channel. Nevertheless, for the majority of…
The notation of mutually unbiased bases(MUB) was first introduced by Ivanovic to reconstruct density matrixes\cite{Ivanovic}. The subject about how to use MUB to analyze, process, and utilize the information of the second moments between…
Two parties observing correlated random variables seek to run an interactive communication protocol. How many bits must they exchange to simulate the protocol, namely to produce a view with a joint distribution within a fixed statistical…
The emergence of nontrivial collective behavior in networks of coupled chaotic maps is investigated by means of a nonlinear mutual prediction method. The resulting prediction error is used to measure the amount of information that a local…
A self-control mechanism for the dynamics of a three-state fully-connected neural network is studied through the introduction of a time-dependent threshold. The self-adapting threshold is a function of both the neural and the pattern…
The persistent mutual information (PMI) is a complexity measure for stochastic processes. It is related to well-known complexity measures like excess entropy or statistical complexity. Essentially it is a variation of the excess entropy so…
Information flow or information transfer is an important concept in dynamical systems which has applications in a wide variety of scientific disciplines. In this study, we show that a rigorous formalism can be established in the context of…
We propose a rationale for experimentally studying the intricate relationship between the rate of information transmission and synchronization level in active networks, applying theoretical results recently proposed. We consider two…
This paper introduces a bilateral matching mechanism to explain why different populations have different levels of cooperation. The traditional game theory assumes that individuals can acquire their neighbor's information without cost after…