Related papers: Minimax R\'enyi Redundancy
Stopping sets play a crucial role in failure events of iterative decoders over a binary erasure channel (BEC). The $\ell$-th stopping redundancy is the minimum number of rows in the parity-check matrix of a code, which contains no stopping…
This paper considers an information bottleneck problem with the objective of obtaining a most informative representation of a hidden feature subject to a R\'enyi entropy complexity constraint. The optimal bottleneck trade-off between…
The Kullback-Leibler (KL) divergence is a foundational measure for comparing probability distributions. Yet in multivariate settings, its single value often obscures the underlying reasons for divergence, conflating mismatches in individual…
We show that the R\'enyi entropies of single particle, extended wave functions for disordered systems contain information about the multifractal spectrum. It is shown for moments of the R\'enyi entropy, $S_{n}$, where $|n|<1$, it is…
What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine…
The matrix-based R\'enyi's entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution. This intriguing property makes it widely applied in statistical…
Given a set $X$ of "empirical" points, whose coordinates are perturbed by errors, we analyze whether it contains redundant information, that is whether some of its elements could be represented by a single equivalent point. If this is the…
Consider a binary word being transmitted through a communication channel that introduces deletable errors where each bit of the word is either retained, flipped, erased or deleted. The simplest code for correcting \emph{all} possible…
There are different inequivalent ways to define the R\'enyi capacity of a channel for a fixed input distribution $P$. In a 1995 paper Csisz\'ar has shown that for classical discrete memoryless channels there is a distinguished such quantity…
We show that for any $\alpha>0$ the R\'enyi entropy of order $\alpha$ is minimized, among all symmetric log-concave random variables with fixed variance, either for a uniform distribution or for a two sided exponential distribution. The…
We propose an extension of the sandwiched R\'enyi relative $\alpha$-entropy to normal positive functionals on arbitrary von Neumann algebras, for the values $\alpha>1$. For this, we use Kosaki's definition of noncommutative $L_p$-spaces…
We study the effects of finite-precision representation of source's probabilities on the efficiency of classic source coding algorithms, such as Shannon, Gilbert-Moore, or arithmetic codes. In particular, we establish the following simple…
A set of classical or quantum states is equivalent to another one if there exists a pair of classical or quantum channels mapping either set to the other one. For dichotomies (pairs of states), this is closely connected to (classical or…
Consider the problem where a statistician in a two-node system receives rate-limited information from a transmitter about marginal observations of a memoryless process generated from two possible distributions. Using its own observations,…
The conditional quantum mutual information $I(A;B|C)$ of a tripartite state $\rho_{ABC}$ is an information quantity which lies at the center of many problems in quantum information theory. Three of its main properties are that it is…
This paper compares a family of methods for characterizing neural feature selectivity with natural stimuli in the framework of the linear-nonlinear model. In this model, the neural firing rate is a nonlinear function of a small number of…
Outage probability and capacity of a class of block-fading MIMO channels are considered with partial channel distribution information. Specifically, the channel or its distribution are not known but the latter is known to belong to a class…
The AWGNC, BSC, and max-fractional pseudocodeword redundancy of a code is defined as the smallest number of rows in a parity-check matrix such that the corresponding minimum pseudoweight is equal to the minimum Hamming distance of the code.…
We show that in any sequence of a general type MMP, the minimal log discrepancy of singularities takes at most finitely many values, and the fibers of all the extremal contractions and flips belong to a bounded family. A key ingredient in…
In this paper, we define a new measure of the redundancy of information from a fault tolerance perspective. The partial information decomposition (PID) emerged last decade as a framework for decomposing the multi-source mutual information…