Related papers: On the Conditional Smooth Renyi Entropy and its Ap…
In this paper, we consider the problem of variable-length source coding allowing errors. The exponential moment of the codeword length is analyzed in the non-asymptotic regime and in the asymptotic regime. Our results show that the smooth…
This paper considers the problem of soft guessing under a logarithmic loss distortion measure while allowing errors. We find an optimal guessing strategy, and derive single-shot upper and lower bounds for the minimal guessing moments as…
We consider the problem of source compression under three different scenarios in the one-shot (non- asymptotic) regime. To be specific, we prove one-shot achievability and converse bounds on the coding rates for distributed source coding,…
This paper provides upper and lower bounds on the optimal guessing moments of a random variable taking values on a finite set when side information may be available. These moments quantify the number of guesses required for correctly…
We consider the problem of guessing the realization of a random variable but under more general Tsallis' non-extensive entropic framework rather than the classical Maxwell-Boltzman-Gibbs-Shannon framework. We consider both the conditional…
This study considers the unconditional smooth R\'{e}nyi entropy, the smooth conditional R\'{e}nyi entropy proposed by Kuzuoka [\emph{IEEE Trans.\ Inf.\ Theory}, vol.~66, no.~3, pp.~1674--1690, 2020], and a new quantity which we term the…
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
Bounds on information combining are entropic inequalities that determine how the information, or entropy, of a set of random variables can change when they are combined in certain prescribed ways. Such bounds play an important role in…
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
We leverage the Gibbs inequality and its natural generalization to R\'enyi entropies to derive closed-form parametric expressions of the optimal lower bounds of $\rho$th-order guessing entropy (guessing moment) of a secret taking values on…
One-shot achievable rate region for source coding when coded side information is available at the decoder (source coding with a helper) is proposed. The achievable region proposed is in terms of conditional smooth max Renyi entropy and…
The Renyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or…
Bounds on information combining are a fundamental tool in coding theory, in particular when analyzing polar codes and belief propagation. They usually bound the evolution of random variables with respect to their Shannon entropy. In recent…
We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent. The minimum entropy required for such a latent is known as common entropy in information theory. We…
A variety of new measures of quantum Renyi mutual information and quantum Renyi conditional entropy have recently been proposed, and some of their mathematical properties explored. Here, we show that the Renyi mutual information attains…
This paper deals with the problem of soft guessing under log-loss distortion (logarithmic loss) that was recently investigated by [Wu and Joudeh, IEEE ISIT, pp. 466--471, 2023]. We extend this problem to soft guessing allowing errors, i.e.,…
In this paper, we analyze the asymptotics of the normalized remaining uncertainty of a source when a compressed or hashed version of it and correlated side-information is observed. For this system, commonly known as Slepian-Wolf source…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…