Related papers: Average entropy of Gaussian mixtures

Within the framework of linear vector Gaussian channels with arbitrary signaling, closed-form expressions for the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system…

Relative entropy serves as a cornerstone concept in quantum information theory. In this work, we study relative entropy of random states from major generic state models of Hilbert-Schmidt and Bures-Hall ensembles. In particular, we derive…

In this paper, we study the problem of learning multi-dimensional Gaussian Mixture Models (GMMs), with a specific focus on model order selection and efficient mixing distribution estimation. We first establish an information-theoretic lower…

This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…

The definition of weighted entropy allows for easy calculation of the entropy of the mixture of measures. In this paper we investigate the problem of equivalent definition of the general entropy function in weighted form. We show that under…

Bayesian methods estimate a measure of uncertainty by using the posterior distribution. One source of difficulty in these methods is the computation of the normalizing constant. Calculating exact posterior is generally intractable and we…

The dissipation of general convex entropies for continuous time Markov processes can be described in terms of backward martingales with respect to the tail filtration. The relative entropy is the expected value of a backward submartingale.…

We study the problem of learning mixtures of $k$ Gaussians in $d$ dimensions. We make no separation assumptions on the underlying mixture components: we only require that the covariance matrices have bounded condition number and that the…

General relationship between mean Boltzmann entropy and Gibbs entropy is established. It is found that their difference is equal to fluctuation entropy, which is a Gibbs-like entropy of macroscopic quantities. The ratio of the fluctuation…

We consider the question of estimating multi-dimensional Gaussian mixtures (GM) with compactly supported or subgaussian mixing distributions. Minimax estimation rate for this class (under Hellinger, TV and KL divergences) is a long-standing…

The envelope of an elliptical Gaussian complex vector, or equivalently, the amplitude or norm of a bivariate normal random vector has application in many weather and signal processing contexts. We explicitly characterize its distribution in…

In this paper we present a method for learning the parameters of a mixture of $k$ identical spherical Gaussians in $n$-dimensional space with an arbitrarily small separation between the components. Our algorithm is polynomial in all…

Mixtures of experts probabilistically divide the input space into regions, where the assumptions of each expert, or conditional model, need only hold locally. Combined with Gaussian process (GP) experts, this results in a powerful and…

We consider convolving a Gaussian of a varying scale epsilon against a Borel measure mu on Euclidean delta-dimensional space. The Lq norm of the result is differentiable in epsilon. We calculate this derivative and show how the upper order…

The ensemble Gaussian mixture filter combines the simplicity and power of Gaussian mixture models with the provable convergence and power of particle filters. The quality of the ensemble Gaussian mixture filter heavily depends on the choice…

We study the statistical behaviour of quantum entanglement in bipartite systems over fermionic Gaussian states as measured by von Neumann entropy. The formulas of average von Neumann entropy with and without particle number constrains have…

We propose a new way to measure the balance between freedom and coherence in a dynamical system and a new measure of its internal variability. Based on the concept of entropy and ideas from neuroscience and information theory, we define…

We discuss the properties of invariant measures corresponding to iterated function systems (IFSs) with place-dependent probabilities and compute their Renyi entropies, generalized dimensions, and multifractal spectra. It is shown that with…

Random matrices formed from i.i.d. standard real Gaussian entries have the feature that the expected number of real eigenvalues is non-zero. This property persists for products of such matrices, independently chosen, and moreover it is…

We consider ensembles of pure Gaussian states parametrized by single-mode marginals and (optionally) specific mode-mode correlations. Such ensembles provide a model for the final states when isolated quantum systems thermalize, as they can…