Related papers: Derivative of BICM Mutual Information
We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that…
In large scale distributed computing systems, communication overhead is one of the major bottlenecks. In the map-shuffle-reduce framework, which is one of the major distributed computing frameworks, the communication load among servers can…
The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $\Pi_{\mathcal{A}}$, on small enough…
This correspondence studies the basic problem of classifications - how to evaluate different classifiers. Although the conventional performance indexes, such as accuracy, are commonly used in classifier selection or evaluation,…
A practical rate-matching system for constructing rate-compatible polar codes is proposed. The proposed polar code circular buffer rate-matching is suitable for transmissions on communication channels that support hybrid automatic repeat…
The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables. One of its important properties is that its expected value is precisely the mutual…
A distribution matcher (DM) encodes a binary input data sequence into a sequence of symbols (codeword) with desired target probability distribution. The set of the output codewords constitutes a codebook (or code) of a DM.…
In the present paper, a simplified two-dimensional magnetic recording (TDMR) channel model is proposed in order to capture the qualitative features of writing and read-back processes of TDMR systems.The proposed channel model incorporates…
We propose a method for MIMO decoding when channel state information (CSI) is unknown to both the transmitter and receiver. The proposed method requires some structure in the transmitted signal for the decoding to be effective, in…
The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission…
The maximal information coefficient (MIC), which measures the amount of dependence between two variables, is able to detect both linear and non-linear associations. However, computational cost grows rapidly as a function of the dataset…
Bit-interleaved coded modulation with iterative decoding (BICM-ID) offers very good error performance over additive white Gaussian noise (AWGN) and fading channels if it uses a wisely designed signal mapping. Further, error performance of…
We consider binary continuous phase modulation (CPM) signals used in some recent low-cost and low-power consumption telecommunications standard. When these signals are generated through a low-cost transmitter, the real modulation index can…
Mutual information (MI) is a useful information-theoretic measure to quantify the statistical dependence between two random variables: $X$ and $Y$. Often, we are interested in understanding how the dependence between $X$ and $Y$ in one set…
Understanding dependencies between variables is critical for interpretability and efficient generation in masked diffusion models (MDMs), yet these models primarily expose marginal conditional distributions and do not explicitly represent…
In the analysis of time series from nonlinear sources, mutual information (MI) is used as a nonlinear statistical criterion for the selection of an appropriate time delay in time delay reconstruction of the state space. MI is a statistic…
Estimating mutual information (MI) is a fundamental task in data science and machine learning. Existing estimators mainly rely on either highly flexible models (e.g., neural networks), which require large amounts of data, or overly…
In this paper we use a well know method in statistics, the $\delta$-method, to provide an asymptotic distribution for the Mutual Information, and construct and independence test based on it. Interesting connections are found with the…
In diffusion-based communication, as for molecular systems, the achievable data rate is very low due to the slow nature of diffusion and the existence of severe inter-symbol interference (ISI). Multiple-input multiple-output (MIMO)…
A new method to measure nonlinear dependence between two variables is described using mutual information to analyze the separate linear and nonlinear components of dependence. This technique, which gives an exact value for the proportion of…