Related papers: Numerical evaluation of Gaussian mixture entropy
The phenomenon of entropy concentration provides strong support for the maximum entropy method, MaxEnt, for inferring a probability vector from information in the form of constraints. Here we extend this phenomenon, in a discrete setting,…
We have discussed the validity of the factorization approximation (FA) and nonextensivity-induced correlation, by using the multivariate $q$-Gaussian probability distribution function (PDF) for $N$-unit independent nonextensive systems. The…
We demonstrate that the shallow water moment equations satisfy an auxiliary entropy conservation law, where the entropy function corresponds to the total energy. Additionally, we show that the classical Newtonian slip friction and Manning…
We propose a new practical adaptive refinement strategy for $hp$-finite element approximations of elliptic problems. Following recent theoretical developments in polynomial-degree-robust a posteriori error analysis, we solve two types of…
We study the multiplicative version of the classical Furstenberg's filtering problem, where instead of the sum $\mathbf{X}+\mathbf{Y}$ one considers the product $\mathbf{X}\cdot \mathbf{Y}$ ($\mathbf{X}$ and $\mathbf{Y}$ are bilateral,…
Estimating the mutual information from samples from a joint distribution is a challenging problem in both science and engineering. In this work, we realize a variational bound that generalizes both discriminative and generative approaches.…
We present a novel method to derive liquid-gas coexisting densities, $\rho^{\pm}(T)$, from grand canonical simulations (without knowledge of $\Tc$ or criticality class). The minima of $ Q_{L}\equiv< m^{2} >_{L}^{2}/< m^{4}>_{L}$ in an…
We derive a general approximate solution to the problem of minimizing the conditional entropy of a qudit-qubit system resulting from a local projective measurement on the qubit, which is valid for general entropic forms and becomes exact in…
We make use of an entropic property to establish a convergence theorem (Main Theorem), which reveals that the conditional entropy measures the asymptotic Gaussianity. As an application, we establish the {\it entropic conditional central…
Gaussian mixture models (GMMs) are ubiquitous in statistical learning, particularly for unsupervised problems. While full GMMs suffer from the overparameterization of their covariance matrices in high-dimensional spaces, spherical GMMs…
For any real numbers $B \ge 1$ and $\delta \in (0, 1)$ and function $f: [0, B] \rightarrow \mathbb{R}$, let $d_{B; \delta} (f) \in \mathbb{Z}_{> 0}$ denote the minimum degree of a polynomial $p(x)$ satisfying $\sup_{x \in [0, B]} \big| p(x)…
The statistical mechanics of self-gravitating systems has not been well understood, and still remains an open question so far. In a previous study by Kang & He, we showed that the fluid approximation may give a clue to further investigate…
We present a unifying approach to the study of entropies in Mathematics, such as measure entropy, topological entropy, algebraic entropy, set-theoretic entropy. We take into account discrete dynamical systems, that is, pairs $(X,T)$, where…
A variational approach is used to calculate free energy and conformational properties in polyelectrolytes. The true bond and Coulomb potentials are approximated by a trial isotropic harmonic energy containing monomer-monomer force constants…
We present an algorithm to estimate the configurational entropy $S$ of a polymer. The algorithm uses the statistics of coincidences among random samples of configurations and is related to the catch-tag-release method for estimation of…
Mixtures of high dimensional Gaussian distributions have been studied extensively in statistics and learning theory. While the total variation distance appears naturally in the sample complexity of distribution learning, it is analytically…
This paper addresses the design of linear and nonlinear stabilization procedures for high-order continuous Galerkin (CG) finite element discretizations of scalar conservation laws. We prove that the standard CG method is entropy…
This paper studies how to learn parameters in diagonal Gaussian mixture models. The problem can be formulated as computing incomplete symmetric tensor decompositions. We use generating polynomials to compute incomplete symmetric tensor…
Low-rank tensor approximations have shown great potential for uncertainty quantification in high dimensions, for example, to build surrogate models that can be used to speed up large-scale inference problems (Eigel et al., Inverse Problems…
We propose a nonparametric density estimator based on the Gaussian process (GP) and derive three novel closed form learning algorithms based on Fisher divergence (FD) score matching. The density estimator is formed by multiplying a base…