Related papers: A Tight Uniform Continuity Bound for Equivocation

We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the…

Using techniques proposed in [Sason, IEEE Trans. Inf. Th. 59, 7118 (2013)] and [Becker, Datta and Jabbour, IEEE Trans. Inf. Th. 69, 4128 (2023)], and based on the results from the latter, we construct a globally optimal continuity bound for…

We prove a tight uniform continuity bound for a family of entropies which includes the von Neumann entropy, the Tsallis entropy and the $\alpha$-R\'enyi entropy, $S_\alpha$, for $\alpha\in (0,1)$. We establish necessary and sufficient…

Answering an open question affirmatively it is shown that every ergodic invariant measure of a mean equicontinuous (i.e. mean-L-stable) system has discrete spectrum. Dichotomy results related to mean equicontinuity and mean sensitivity are…

Uncertainty relations are central to quantum physics. While they were originally formulated in terms of variances, they have later been successfully expressed with entropies following the advent of Shannon information theory. Here, we…

Entropic uncertainty relations place nontrivial lower bounds to the sum of Shannon information entropies for noncommuting observables. Here we obtain a novel lower bound on the entropy sum for general pairs of observables in…

The entropy region is a fundamental object in information theory. An outer bound for the entropy region is defined by a minimal set of Shannon-type inequalities called elemental inequalities also referred to as the Shannon region. This…

We prove a sharp bound between sampling numbers and entropy numbers in the uniform norm for bounded convex sets of bounded functions.

In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…

This paper develops sharp bounds on moments of sums of k-wise independent bounded random variables, under constrained average variance. The result closes the problem addressed in part in the previous works of Schmidt et al. and Bellare,…

We give the proof of a tight lower bound on the probability that a binomial random variable exceeds its expected value. The inequality plays an important role in a variety of contexts, including the analysis of relative deviation bounds in…

To overcome the drawbacks of Shannon's entropy, the concept of cumulative residual and past entropy has been proposed in the information theoretic literature. Furthermore, the Shannon entropy has been generalized in a number of different…

This paper is prepared as a contribution to the proceedings after the 12th ICSSUR/Feynfest Conference held in Foz do Iguacu (Brazil) from 2 to 6 May 2011. In the first part I briefy report the topic of entropic uncertainty relations for…

Shannon entropy for discrete distributions is a fundamental and widely used concept, but its continuous analogue, known as differential entropy, lacks essential properties such as positivity and compatibility with the discrete case. In this…

We show that a proper expression of the uncertainty relation for a pair of canonically-conjugate continuous variables relies on entropy power, a standard notion in Shannon information theory for real-valued signals. The resulting…

We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…

The Renyi entropy plays an essential role in quantum information theory. We study the continuity estimation of the Renyi entropy. An inequality relating the Renyi entropy difference of two quantum states to their trace norm distance is…

The paper presents an extension of Shannon fuzzy entropy for intuitionistic fuzzy one. Firstly, we presented a new formula for calculating the distance and similarity of intuitionistic fuzzy information. Then, we constructed measures for…

Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…

I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss…