Related papers: Arimoto-R\'enyi Conditional Entropy and Bayesian $…
Machine learning models have traditionally been developed under the assumption that the training and test distributions match exactly. However, recent success in few-shot learning and related problems are encouraging signs that these models…
We examine the minimum entropy coupling problem, where one must find the minimum entropy variable that has a given set of distributions $S = \{p_1, \dots, p_m \}$ as its marginals. Although this problem is NP-Hard, previous works have…
Given an observed random variable, consider the problem of recovering its distribution among a family of candidate ones. The two-point inequality, Fano's lemma and more recently an inequality due to Venkataramanan and Johnson (2018) allow…
We revisit the outlier hypothesis testing framework of Li \emph{et al.} (TIT 2014) and derive fundamental limits for the optimal test under the generalized Neyman-Pearson criterion. In outlier hypothesis testing, one is given multiple…
We define and investigate a notion of entropy for quantum error correcting codes. The entropy of a code for a given quantum channel has a number of equivalent realisations, such as through the coefficients associated with the Knill-Laflamme…
Consider the problem of detecting one of M i.i.d. Gaussian signals corrupted in white Gaussian noise. Conventionally, matched filters are used for detection. We first show that the outputs of the matched filter form a set of asymptotically…
In this paper, we analyze the asymptotics of the normalized remaining uncertainty of a source when a compressed or hashed version of it and correlated side-information is observed. For this system, commonly known as Slepian-Wolf source…
We study the distribution of the order of a random permutation of $[n]$ through the lens of R\'enyi entropy. In particular, we obtain an asymptotic for the R\'enyi $q$-entropy of the order in the full range $1 \leq q \leq \infty$. For $q >…
We show how one can demonstrate continuous-variable Einstein-Podolsky-Rosen (EPR) steering without needing to characterize entire measurement probability distributions. To do this, we develop a modified Fano inequality useful for discrete…
Uniform continuity bounds on entropies are generally expressed in terms of a single distance measure between a pair of probability distributions or quantum states, typically, the total variation distance or trace distance. However, if an…
An uncertainty relation for the R\'enyi entropies of conjugate quantum observables is used to obtain a strong Heisenberg limit of the form ${\rm RMSE} \geq f(\alpha)/(\langle N\rangle+\frac12)$, bounding the root mean square error of any…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…
We investigate the uncertainty principle for two successive projective measurements in terms of R\'enyi entropy based on a single quantum system. Our results cover a large family of the entropy (including the Shannon entropy) uncertainty…
Within a Bayesian decision theoretic framework we investigate some asymptotic optimality properties of a large class of multiple testing rules. A parametric setup is considered, in which observations come from a normal scale mixture model…
Lower bounds involving $f$-divergences between the underlying probability measures are proved for the minimax risk in estimation problems. Our proofs just use simple convexity facts. Special cases and straightforward corollaries of our…
Quantum key distribution requires tight and reliable bounds on the secret key rate to ensure robust security. This is particularly so for the regime of finite block sizes, where the optimization of generalized R\'enyi entropic quantities is…
Behavior of the entropy numbers of classes of multivariate functions with mixed smoothness is studied here. This problem has a long history and some fundamental problems in the area are still open. The main goal of this paper is to develop…
We seek an entropy estimator for discrete distributions with fully empirical accuracy bounds. As stated, this goal is infeasible without some prior assumptions on the distribution. We discover that a certain information moment assumption…
The problem of channel coding with the erasure option is revisited for discrete memoryless channels. The interplay between the code rate, the undetected and total error probabilities is characterized. Using the information spectrum method,…
We investigate the R\'enyi entropy of independent sums of integer valued random variables through Fourier theoretic means, and give sharp comparisons between the variance and the R\'enyi entropy, for Poisson-Bernoulli variables. As…