Related papers: The Entropy of Floating-Point Numbers
Gaussian mixture distributions are commonly employed to represent general probability distributions. Despite the importance of using Gaussian mixtures for uncertainty estimation, the entropy of a Gaussian mixture cannot be calculated…
The quantum relative entropy is a fundamental quantity in quantum information science, characterizing the distinguishability between two quantum states. However, this quantity is not additive in general for correlated quantum states,…
We consider the one-parameter family of interval maps arising from generalized continued fraction expansions known as alpha-continued fractions. For such maps, we perform a numerical study of the behaviour of metric entropy as a function of…
We apply a common measure of randomness, the entropy, in the context of iterated functions on a finite set with n elements. For a permutation, it turns out that this entropy is asymptotically (for a growing number of iterations) close to…
In this article we continue to study the concept of entropy introduced in [4], [15]-[17]. We calculate entropy for a wider class of finite-dimensional operators in comparison with [15]. We also approximate the entropy of a unitary operator…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…
The maximum entropy principle is a powerful tool for solving underdetermined inverse problems. This paper considers the problem of discretizing a continuous distribution, which arises in various applied fields. We obtain the approximating…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
Quantum physics, despite its observables being intrinsically of a probabilistic nature, does not have a quantum entropy assigned to them. We propose a quantum entropy that quantify the randomness of a pure quantum state via a conjugate pair…
We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and…
The design of embedded control systems is mainly done with model-based tools such as Matlab/Simulink. Numerical simulation is the central technique of development and verification of such tools. Floating-point arithmetic, that is well-known…
It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…
This document expands upon the relationship between discrete and continuous entropy given in (Phys. Rev. Lett. 110 130407), \Violating Continuous Variable Einstein-Podolsky-Rosen Steering with Discrete Measurements". We provide a detailed…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
This paper demonstrates that basic statistics (mean, variance) of the logarithm of the variate itself can be used in the calculation of differential entropy among random variables known to be multiples and powers of a common underlying…
In this paper relations among some kinds of cumulative entropies and moments of order statistics are presented. By using some characterizations and the symmetry of a non negative and absolutely continuous random variable X, lower and upper…
The time decay of fully discrete finite-volume approximations of porous-medium and fast-diffusion equations with Neumann or periodic boundary conditions is proved in the entropy sense. The algebraic or exponential decay rates are computed…