Related papers: Statistical Estimation of Conditional Shannon Entr…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
This paper generalizes recent proposals of density forecasting models and it develops theory for this class of models. In density forecasting, the density of observations is estimated in regions where the density is not observed.…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
Determining the strength of non-linear statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual…
Entropic uncertainty relations for the position and momentum within the generalized uncertainty principle are examined. Studies of this principle are motivated by the existence of a minimal observable length. Then the position and momentum…
This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…
We describe five types of results concerning information and concentration of discrete random variables, and relationships between them, motivated by their counterparts in the continuous case. The results we consider are information…
This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a…
Entropic measures provide analytic tools to help us understand correlation in quantum systems. In our previous work, we calculated linear entropy and von Neumann entropy as entanglement measures for the ground state and lower lying excited…
The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown.…
Entropic uncertainty relations place nontrivial lower bounds to the sum of Shannon information entropies for noncommuting observables. Here we obtain a novel lower bound on the entropy sum for general pairs of observables in…
Recently established, directed dependence measures for pairs $(X,Y)$ of random variables build upon the natural idea of comparing the conditional distributions of $Y$ given $X=x$ with the marginal distribution of $Y$. They assign pairs…
The minimum error entropy (MEE) criterion has been successfully used in fields such as parameter estimation, system identification and the supervised machine learning. There is in general no explicit expression for the optimal MEE estimate…
The entropy region is a fundamental object in information theory. An outer bound for the entropy region is defined by a minimal set of Shannon-type inequalities called elemental inequalities also referred to as the Shannon region. This…
The internal disorder of hydrogenic Rydberg atoms as contained in their position and momentum probability densities is examined by means of the following information-theoretic spreading quantities: the radial and logarithmic expectation…
In this paper, we provide the R\'enyi entropy and complexity measure for a novel, flexible class of skew-gaussian distributions and their related families, as a characteristic form of the skew-gaussian Shannon entropy. We give closed…
We study in details the bias and variance of the entropy estimator proposed by Kozachenko and Leonenko for a large class of densities on $\mathbb{R}^d$. We then use the work of Bickel and Breiman to prove a central limit theorem in…
Configurational entropy has been revealed as a reliable method for constraining some parameters of a given model [Phys. Rev. D \textbf{92} (2015) 126005, Eur. Phys. J. C \textbf{76} (2016) 100]. In this letter we calculate the…
The R\'enyi and Shannon entropies are information-theoretic measures which have enabled to formulate the position-momentum uncertainty principle in a much more adequate and stringent way than the (variance-based) Heisenberg-like relation.…