Related papers: $H(X)$ vs. $H(f(X))$

In this paper we provide a method to obtain tight lower bounds on the minimum redundancy achievable by a Huffman code when the probability distribution underlying an alphabet is only partially known. In particular, we address the case where…

A lower bound on the probability $P(0<X<\delta)$ for all real $\delta>0$ and all random variables $X$ with log-concave p.d.f.'s such that $EX=0$ and $EX^2=1$ is obtained.

We show a direct relationship between the variance and the differential entropy for subclasses of symmetric and asymmetric unimodal distributions by providing an upper bound on variance in terms of entropy power. Combining this bound with…

In this paper a numerical method is presented, which finds a lower bound for the mutual information between a binary and an arbitrary finite random variable with joint distributions that have a variational distance not greater than a known…

Entropy in thermodynamics is an extensive quantity, whereas standard methods in statistical mechanics give rise to a non-extensive expression for the entropy. This discrepancy is often seen as a sign that basic formulas of statistical…

We derive a number of equivalent criterions for the variable exponent Hardy type inequality |\frac{1}{x}\int_{0}^{x}f(t)dt|_{L^{p(.)}(0,1)}\leq C|f|_{L^{p(.)}(0,1)}; f\geq 0. to hold, whenever the exponent $p:(0,1)\to (1,\infty)$ is…

We compare an entropy estimator $H_z$ recently discussed in [10] with two estimators $H_1$ and $H_2$ introduced in [6][7]. We prove the identity $H_z \equiv H_1$, which has not been taken into account in [10]. Then, we prove that the…

Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…

A central object in optimal stopping theory is the single-choice prophet inequality for independent, identically distributed random variables: Given a sequence of random variables $X_1,\dots,X_n$ drawn independently from a distribution $F$,…

We analyze entropic uncertainty relations for two orthogonal measurements on a $N$-dimensional Hilbert space, performed in two generic bases. It is assumed that the unitary matrix $U$ relating both bases is distributed according to the Haar…

We propose some new results on the comparison of the minimum or maximum order statistic from a random number of non-identical random variables. Under the non-identical set-up, with certain conditions, we prove that random minimum (maximum)…

Let $(X,\phi)$ be a compact flow without fixed points. We define the packing topological entropy $h_{\mathrm{top}}^P(\phi,K)$ on subsets of $X$ through considering all the possible reparametrizations of time. For fixed-point free flows, we…

Power law distributions are widely observed in chemical physics, geophysics, biology, and beyond. The independent variable x of these distributions has an obligatory lower bound and in many cases also an upper bound. Estimating these bounds…

We describe a method to computationally estimate the probability density function of a univariate random variable by applying the maximum entropy principle with some local conditions given by Gaussian functions. The estimation errors and…

Subentropy is an entropy-like quantity that arises in quantum information theory; for example, it provides a tight lower bound on the accessible information for pure state ensembles, dual to the von Neumann entropy upper bound in Holevo's…

The Hamiltonian actions for extreme and non-extreme black holes are compared and contrasted and a simple derivation of the lack of entropy of extreme black holes is given. In the non-extreme case the wave function of the black hole depends…

The $f$-invariant is a notion of entropy for probability-measure-preserving actions of free groups. We show it is invariant under bounded orbit-equivalence.

Motivated by the approach of random linear codes, a new distance in the vector space over a finite field is defined as the logarithm of the "surface area" of a Hamming ball with radius being the corresponding Hamming distance. It is named…

Although there is an extensive literature on the upper bound for cumulative standard normal distribution, there are relatively not sharp for all values of the interested argument x. The aim of this paper is to establish a sharp upper bound…

We show that absolutely minimizing functions relative to a convex Hamiltonian $H:\mathbb{R}^n \to \mathbb{R}$ are uniquely determined by their boundary values under minimal assumptions on $H.$ Along the way, we extend the known equivalences…