Related papers: A new entropy power inequality for integer-valued …
Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimensions. This paper presents estimators of entropy via inference…
Given two jointly distributed random variables $(X,Y)$, a functional representation of $X$ is a random variable $Z$ independent of $Y$, and a deterministic function $g(\cdot, \cdot)$ such that $X=g(Y,Z)$. The problem of finding a minimum…
Two estimates for the inverse binary entropy function are derived using the property of information entropy to estimate combinatorics of sequences as well as related formulas from population genetics for the effective number of alleles. The…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
Statistical mechanics is generalized on the basis of an additive information theory for incomplete probability distributions. The incomplete normalization $\sum_{i=1}^wp_i^q=1$ is used to obtain generalized entropy $S=-k\sum_{i=1}^wp_i^q\ln…
Hwang's quasi-power theorem asserts that a sequence of random variables whose moment generating functions are approximately given by powers of some analytic function is asymptotically normally distributed. This theorem is generalised to…
In this study, the quantum R\'{e}nyi entropy power inequality of order $p>1$ and power $\kappa$ is introduced as a quantum analog of the classical R\'{e}nyi-$p$ entropy power inequality. To derive this inequality, we first exploit the…
In 1994, Jim Massey proposed the guessing entropy as a measure of the difficulty that an attacker has to guess a secret used in a cryptographic system, and established a well-known inequality between entropy and guessing entropy. Over 15…
Building on the recent work of Johnson (2007) and Yu (2008), we prove that entropy is a concave function with respect to the thinning operation T_a. That is, if X and Y are independent random variables on Z_+ with ultra-log-concave…
We study an information analogue of infinitely divisible probability distributions, where the i.i.d. sum is replaced by the joint distribution of an i.i.d. sequence. A random variable $X$ is called informationally infinitely divisible if,…
In this article we find some properties of certain types of entropies of a natural number. Also, regarding the entropy H of a natural number, introduced by Minculete and Pozna, we generalize this notion for ideals and we find some of its…
We offer a new point of view on the (Modified) Log-Sobolev inequality and lower bounds on the Ricci-curvature in the setting where the dynamics are obtained as the limit of Markov processes. In this setting, the large deviation rate…
In this paper, we derive entropy estimates for a class of schemes for the Euler equations which present the following features: they are based on the internal energy equation (eventually with a positive corrective term at the righ-hand-side…
We show that a proper expression of the uncertainty relation for a pair of canonically-conjugate continuous variables relies on entropy power, a standard notion in Shannon information theory for real-valued signals. The resulting…
This paper considers the estimation of Shannon entropy for discrete distributions with countably infinite support. While minimax rates for finite-support distributions are established, infinite-support distributions present distinct…
Recent literature in the last Maximum Entropy workshop introduced an analogy between cumulative probability distributions and normalized utility functions. Based on this analogy, a utility density function can de defined as the derivative…
This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…
We obtain inequalities involving the entropy of a positive integer and the divergence of two positive integers, respectively the entropy of an ideal and the divergence of two ideals in a ring of algebraic integers. Among the important…
In this paper we apply the entropy principle to the relativistic version of the differential equations describing a standard fluid flow, that is, the equations for mass, momentum, and a system for the energy matrix. These are the second…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…