Related papers: Statistical Properties of the Entropy from Ordinal…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
This study investigates entropy's potential for analyzing scientific research patterns across disciplines. Originating from thermodynamics, entropy now measures uncertainty and diversity in information systems. We examine Shannon Entropy,…
For statistical systems that violate one of the four Shannon-Khinchin axioms, entropy takes a more general form than the Boltzmann-Gibbs entropy. The framework of superstatistics allows one to formulate a maximum entropy principle with…
For probability measures on countable spaces we derive distributional limits for empirical entropic optimal transport quantities. More precisely, we show that the empirical optimal transport plan weakly converges to a centered Gaussian…
Thermodynamic uncertainty relations reveal a fundamental trade-off between the precision of a trajectory observable and entropy production, where the uncertainty of the observable is quantified by its variance. In information theory,…
Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…
A method for estimating the Shannon differential entropy of multidimensional random variables using independent samples is described. The method is based on decomposing the distribution into a product of the marginal distributions and the…
In statistical analysis, quantifying uncertainties through measures such as entropy, extropy, varentropy, and varextropy is of fundamental importance for understanding distribution functions. This paper investigates several properties of…
This letter reports two moment extensions of the entropy of a distribution. By understanding the traditional entropy as the average of the original distribution up to a random variable transformation, the traditional moments equation become…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
Entropy metrics (for example, permutation entropy) are nonlinear measures of irregularity in time series (one-dimensional data). Some of these entropy metrics can be generalised to data on periodic structures such as a grid or lattice…
Superstatistics describes statistical systems that behave like superpositions of different inverse temperatures $\beta$, so that the probability distribution is $p(\epsilon_i) \propto \int_{0}^{\infty} f(\beta) e^{-\beta \epsilon_i}d\beta$,…
We address the generalized uncertainty principle in scenarios of successive measurements. Uncertainties are characterized by means of generalized entropies of both the R\'{e}nyi and Tsallis types. Here, specific features of measurements of…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
From the output produced by a memoryless deletion channel from a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
We describe an approach to improving model fitting and model generalization that considers the entropy of distributions of modelling residuals. We use simple simulations to demonstrate the observational signatures of overfitting on ordered…
In many applications, the probability density function is subject to experimental errors. In this work the continuos dependence of a class of generalized entropies on the experimental errors is studied. This class includes the C. Shannon,…
Two families of dependence measures between random variables are introduced. They are based on the R\'enyi divergence of order $\alpha$ and the relative $\alpha$-entropy, respectively, and both dependence measures reduce to Shannon's mutual…
In this work, we develop a general method for estimating the Shannon entropy of a bidimensional sequence based on the extrapolation of block entropies. We apply this method to analyse the spatial configurations of cities of different…
Understanding the way in which random entities interact is of key interest in numerous scientific fields. This can range from a full characterization of the joint distribution to single scalar summary statistics. In this work we identify a…