Related papers: Average entropy of Gaussian mixtures

The entropy computation of Gaussian mixture distributions with a large number of components has a prohibitive computational complexity. In this paper, we propose a novel approach exploiting the sphere decoding concept to bound and…

It is by now well known that the Boltzmann-Gibbs (BG) entropy $S_{BG}=-k\sum_{i=1}^W p_i \ln p_i$ can be usefully generalized into the entropy $S_q=k (1-\sum_{i=1}^Wp_i^{q}) / (q-1)$ ($q\in \mathcal{R}; S_1=S_{BG}$). Microscopic dynamics…

Differential entropy and log determinant of the covariance matrix of a multivariate Gaussian distribution have many applications in coding, communications, signal processing and statistical inference. In this paper we consider in the high…

Averaging is an important method to extract effective macroscopic dynamics from complex systems with slow modes and fast modes. This article derives an averaged equation for a class of stochastic partial differential equations without any…

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential…

We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…

Estimation of the covariance matrix has attracted a lot of attention of the statistical research community over the years, partially due to important applications such as Principal Component Analysis. However, frequently used empirical…

So-called average subsystem entropies are defined by first taking partial traces over some pure state to define density matrices, then calculating the subsystem entropies, and finally averaging over the pure states to define the average…

In this paper we consider the space of those probability distributions which maximize the $q$-R\'enyi entropy. These distributions have the same parameter space for every $q$, and in the $q=1$ case these are the normal distributions. Some…

We consider estimating the parameters of a Gaussian mixture density with a given number of components best representing a given set of weighted samples. We adopt a density interpretation of the samples by viewing them as a discrete Dirac…

Let $K = \{0,1,...,q-1\}$. We use a special class of translation invariant measures on $K^\mathbb{Z}$ called algebraic measures to study the entropy rate of a hidden Markov processes. Under some irreducibility assumptions of the Markov…

Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…

We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…

We present randomized algorithms for sampling the standard Gaussian distribution restricted to a convex set and for estimating the Gaussian measure of a convex set, in the general membership oracle model. The complexity of integration is…

In this article, we are proposing a closed-form solution for the capacity of the single quantum channel. The Gaussian distributed input has been considered for the analytical calculation of the capacity. In our previous couple of papers, we…

We investigate the average entropy of a subsystem within a global unitary orbit of a given mixed bipartite state in the finite-dimensional space. Without working out the closed-form expression of such average entropy for the mixed state…

Gaussian mixture filters for nonlinear systems usually rely on severe approximations when calculating mixtures in the prediction and filtering step. Thus, offline approximations of noise densities by Gaussian mixture densities to reduce the…

The eigenvalue density for members of the Gaussian orthogonal and unitary ensembles follows the Wigner semi-circle law. If the Gaussian entries are all shifted by a constant amount c/Sqrt(2N), where N is the size of the matrix, in the large…

We study the complexity of learning mixtures of separated Gaussians with common unknown bounded covariance matrix. Specifically, we focus on learning Gaussian mixture models (GMMs) on $\mathbb{R}^d$ of the form $P= \sum_{i=1}^k w_i…

Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is…