Related papers: An entropy functional bounded from above by one
Given a universe of discourse X-a domain of possible outcomes-an experiment may consist of selecting one of its elements, subject to the operation of chance, or of observing the elements, subject to imprecision. A priori uncertainty about…
This paper derives bounds for two omnipresent information theoretic measures, the Shannon entropy and its complementary dual, the extropy. Based on a large size data set from a logconcave model, the said bounds are obtained for the entropy…
We introduce unbiased estimators for the Shannon entropy and the class number, in the situation that we are able to take sequences of independent samples of arbitrary length.
A probabilistic source is defined as the set of infinite words (over a given denumerable alphabet) endowed with a probability $\mu$. The paper deals with general binary sources where the distribution of any symbol (0 or 1) may depend on an…
We study convexity properties of R\'{e}nyi entropy as function of $\alpha>0$ on finite alphabets. We also describe robustness of the R\'{e}nyi entropy on finite alphabets, and it turns out that the rate of respective convergence depends on…
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits…
The distortion-rate performance of certain randomly-designed scalar quantizers is determined. The central results are the mean-squared error distortion and output entropy for quantizing a uniform random variable with thresholds drawn…
This work contains two single-letter upper bounds on the entropy rate of a discrete-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large…
The minimal set of Shannon-type inequalities (referred to as elemental inequalities), plays a central role in determining whether a given inequality is Shannon-type. Often, there arises a situation where one needs to check whether a given…
Misclassification detection is an important problem in machine learning, as it allows for the identification of instances where the model's predictions are unreliable. However, conventional uncertainty measures such as Shannon entropy do…
We address a recent conjecture stated by Z. Van Herstraeten and N.J. Cerf. They claim that the Shannon entropy for positive Wigner functions is bounded below by a positive constant, which can be attained only by Gaussian pure states. We…
A lower bound on the R\'enyi differential entropy of a sum of independent random vectors is demonstrated in terms of rearrangements. For the special case of Boltzmann-Shannon entropy, this lower bound is better than that given by the…
This paper concerns the behavior of eigenfunctions of quantized cat maps and in particular their supremum norm. We observe that for composite integer values of N, the inverse of Planck's constant, some of the desymmetrized eigenfunctions…
Previously derived expressions for the characteristic function of work performed on a quantum system by a classical external force are generalized to arbitrary initial states of the considered system and to Hamiltonians with degenerate…
The concept of entropy connects the number of possible configurations with the number of variables in large stochastic systems. Independent or weakly interacting variables render the number of configurations scale exponentially with the…
The estimation of entropy rates for stationary discrete-valued stochastic processes is a well studied problem in information theory. However, estimating the entropy rate for stationary continuous-valued stochastic processes has not received…
Compressed Counting (CC) [22] was recently proposed for estimating the ath frequency moments of data streams, where 0 < a <= 2. CC can be used for estimating Shannon entropy, which can be approximated by certain functions of the ath…
We give a distribution-dependent concentration inequality for functions of independent variables. The result extends Bernstein's inequality from sums to more general functions, whose variation in any argument does not depend too much on the…
This paper addresses the correspondence between linear inequalities of Shannon entropy and differential entropy for sums of independent group-valued random variables. We show that any balanced (with the sum of coefficients being zero)…
The notion of probability density for a random function is not as straightforward as in finite-dimensional cases. While a probability density function generally does not exist for functional data, we show that it is possible to develop the…