Related papers: Average entropy of Gaussian mixtures
Gaussian mixture distributions are commonly employed to represent general probability distributions. Despite the importance of using Gaussian mixtures for uncertainty estimation, the entropy of a Gaussian mixture cannot be calculated…
We present a series of closed-form maximum entropy upper bounds for the differential entropy of a continuous univariate random variable and study the properties of that series. We then show how to use those generic bounds for upper bounding…
The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs be obtained from the data sample itself. We propose a semi-parametric…
Studying the typical entanglement entropy of a bipartite system when averaging over different ensembles of pure quantum states has been instrumental in different areas of physics, ranging from many-body quantum chaos to black hole…
We develop an approximation method for the differential entropy $h(\mathbf{X})$ of a $q$-component Gaussian mixture in $\mathbb{R}^n$. We provide two examples of approximations using our method denoted by…
We derive the spectral density of the equiprobable mixture of two random density matrices of a two-level quantum system. We also work out the spectral density of mixture under the so-called quantum addition rule. We use the spectral…
A $q$-Gaussian measure is a generalization of a Gaussian measure. This generalization is obtained by replacing the exponential function with the power function of exponent $1/(1-q)$ ($q\neq 1$). The limit case $q=1$ recovers a Gaussian…
This paper deals with the estimation of one-dimensional Gaussian mixture. Given a set of observations of a K-component Gaussian mixture, we focus on the estimation of the component expectations. The number of components is supposed to be…
The classical Gaussian ensembles of random matrices can be constructed by maximizing Boltzmann-Gibbs-Shannon's entropy, S_{BGS} = - \int d{\bf H} [P({\bf H})] \ln [P({\bf H})], with suitable constraints. Here we construct and analyze…
The diagonal entropy was introduced as a good entropy candidate especially for isolated quantum systems out of equilibrium. Here we present an analytical calculation of the average diagonal entropy for systems undergoing unitary evolution…
We have discussed the Tsallis entropy in finite $N$-unit nonextensive systems, by using the multivariate $q$-Gaussian probability distribution functions (PDFs) derived by the maximum entropy methods with the normal average and the…
The entropy change that occurs upon mixing two fluids has remained an intriguing topic since the dawn of statistical mechanics. In this work, we generalize the grand-isobaric ensemble to mixtures, and develop a Monte Carlo algorithm for the…
A symmetric random variable is called a Gaussian mixture if it has the same distribution as the product of two independent random variables, one being positive and the other a standard Gaussian random variable. Examples of Gaussian mixtures…
The entropy of an ergodic finite-alphabet process can be computed from a single typical sample path x_1^n using the entropy of the k-block empirical probability and letting k grow with $n$ roughly like log n. We further assume that the…
We calculate Gaussian averages of arbitrary exponentials of the matrix variable $X$ with the help of superintegrability, which provides explicit expressions for Schur averages. As in the simpler cases the answer is expressed in terms of…
Following [1], the aim of this paper is to analyze the relative weighted entropy involving the central moments weight functions. We compare the standard relative entropy with the weighted case in two particular forms of Gaussian…
We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We…
Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and…
This work provides a computationally efficient and statistically consistent moment-based estimator for mixtures of spherical Gaussians. Under the condition that component means are in general position, a simple spectral decomposition…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…