Related papers: Average entropy of Gaussian mixtures
The entropy production of a nonequilibrium system with broken detailed balance is a random variable whose mean value is nonnegative. Among the total entropy production, the housekeeping entropy production is associated with the heat…
Gaussian processes are a key component of many flexible statistical and machine learning models. However, they exhibit cubic computational complexity and high memory constraints due to the need of inverting and storing a full covariance…
Constructions in type-driven compositional distributional semantics associate large collections of matrices of size $D$ to linguistic corpora. We develop the proposal of analysing the statistical characteristics of this data in the…
R\'enyi entropy of order \alpha is a general measure of entropy. In this paper we derive estimations for the R\'enyi entropy of the mixture of sources in terms of the entropy of the single sources. These relations allow to compute the…
We present a derivation of the Von Neumann entropy and mutual information of arbitrary two--mode Gaussian states, based on the explicit determination of the symplectic eigenvalues of a generic covariance matrix. The key role of the…
Mixtures of Gaussian factors are powerful tools for modeling an unobserved heterogeneous population, offering - at the same time - dimension reduction and model-based clustering. Unfortunately, the high prevalence of spurious solutions and…
We present a simple Coulomb gas method to calculate analytically the probability of rare events where the maximum eigenvalue of a random matrix is much larger than its typical value. The large deviation function that characterizes this…
We prove a general independent equidistribution result for Gauss sums associated to $n$ monomials in $r$ variable multiplicative characters over a finite field, which generalizes several previous equidistribution results for Gauss and…
It is observed that the entropy reduction (the information gain in the initial terminology) of an efficient (ideal or pure) quantum measurement coincides with the generalized quantum mutual information of a q-c channel mapping an a priori…
We have studied finite $N$-body $D$-dimensional nonextensive ideal gases and harmonic oscillators, by using the maximum-entropy methods with the $q$- and normal averages ($q$: the entropic index). The validity range, specific heat and…
We propose a method for computing the Kolmogorov-Sinai (KS) entropy of chaotic systems. In this method, the KS entropy is expressed as a statistical average over the canonical ensemble for a Hamiltonian with many ground states. This…
We analyze the distribution that extremizes a linear combination of the Boltzmann--Gibbs entropy and the nonadditive $q$-entropy. We show that this distribution can be expressed in terms of a Lambert function. Both the entropic functional…
Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. In this manuscript, we characterise the learning of a mixture of $K$ Gaussians with generic means…
We consider the problem of estimating the common mean of independently sampled data, where samples are drawn in a possibly non-identical manner from symmetric, unimodal distributions with a common mean. This generalizes the setting of…
We produce a series of results extending information-theoretical inequalities (discussed by Dembo--Cover--Thomas in 1989-1991) to a weighted version of entropy. The resulting inequalities involve the Gaussian weighted entropy; they imply a…
We consider an isomorphism invariant for measure-preserving systems - types of generalized entropy convergence rates. We show the connections of this invariant with the types of Shannon entropy convergence rates. In the case when they…
This paper proposes a novel approach to the statistical characterization of non-central complex Gaussian quadratic forms (CGQFs). Its key strategy is the generation of an auxiliary random variable (RV) that converges in distribution to the…
Coincidence probabilities, which yield Renyi entropies, are investigated in a generalized Gaussian model, which includes interparticle correlations
We establish a sharp estimate for a minimal number of binary digits (bits) needed to represent all bounded total generalized variation functions taking values in a general totally bounded metric space $(E,\rho)$ up to an accuracy of…
The theory of large deviations constitutes a mathematical cornerstone in the foundations of Boltzmann-Gibbs statistical mechanics, based on the additive entropy $S_{BG}=- k_B\sum_{i=1}^W p_i \ln p_i$. Its optimization under appropriate…