Related papers: Numerical evaluation of Gaussian mixture entropy
Let $\bf{x}$ be a random variable with density $\rho(x)$ taking values in ${\mathbb R}^d$. We are interested in finding a representation for the shape of $\rho(x)$, i.e. for the orbit $\{ \rho(g\cdot x) | g\in E(d) \}$ of $\rho$ under the…
We propose to use a hybridizable discontinuous Galerkin (HDG) method combined with the continuous Galerkin (CG) method to approximate Maxwell's equations. We make two contributions in this paper. First, even though there are many papers…
A tracer sample in a gravitational potential, starting from a generic initial condition, phase-mixes towards a stationary state. This evolution is accompanied by an entropy increase, and the final state is characterized by a distribution…
We describe a method to computationally estimate the probability density function of a univariate random variable by applying the maximum entropy principle with some local conditions given by Gaussian functions. The estimation errors and…
The quantum query complexity of Boolean matrix multiplication is typically studied as a function of the matrix dimension, n, as well as the number of 1s in the output, \ell. We prove an upper bound of O (n\sqrt{\ell}) for all values of…
In this paper, we study the problem of learning multi-dimensional Gaussian Mixture Models (GMMs), with a specific focus on model order selection and efficient mixing distribution estimation. We first establish an information-theoretic lower…
We establish a dimension-free improvement of Talagrand's Gaussian transport-entropy inequality, under the assumption that the measures satisfy a Poincar\'e inequality. We also study stability of the inequality, in terms of relative entropy,…
Estimating the number of components is a fundamental challenge in unsupervised learning, particularly when dealing with high-dimensional data with many components or severely imbalanced component sizes. This paper addresses this challenge…
This work establishes computable bounds between f-divergences for probability measures within a generalized quasi-$\varepsilon_{(M,m)}$-neighborhood framework. We make the following key contributions. (1) a unified characterization of local…
The Expectation-Maximization (EM) algorithm is a widely used method for maximum likelihood estimation in models with latent variables. For estimating mixtures of Gaussians, its iteration can be viewed as a soft version of the k-means…
We present a novel approach to approximate Gaussian and mixture-of-Gaussians filtering. Our method relies on a variational approximation via a gradient-flow representation. The gradient flow is derived from a Kullback--Leibler discrepancy…
Within the framework of linear vector Gaussian channels with arbitrary signaling, closed-form expressions for the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system…
Given $k$-uniform hypergraphs $G$ and $H$ on $n$ vertices with densities $p$ and $q$, their relative discrepancy is defined as $\hbox{disc}(G,H)=\max\big||E(G')\cap E(H')|-pq\binom{n}{k}\big|$, where the maximum ranges over all pairs…
In recent years, several algorithms, which approximate matrix decomposition, have been developed. These algorithms are based on metric conservation features for linear spaces of random projection types. We show that an i.i.d sub-Gaussian…
We introduce a systematic method for constructing polytope approximations to the quantum set in a variety of device-independent quantum random number generation (DI-QRNG) protocols. Our approach relies on two general-purpose algorithms that…
In this paper, we derive an a-posteriori error indicator for the Generalized Multiscale Finite Element Method (GMsFEM) framework. This error indicator is further used to develop an adaptive enrichment algorithm for the linear elliptic…
In this paper, a distance between the Gaussian Mixture Models(GMMs) is obtained based on an embedding of the K-component Gaussian Mixture Model into the manifold of the symmetric positive definite matrices. Proof of embedding of K-component…
We study the approximability of general convex sets in $\mathbb{R}^n$ by intersections of halfspaces, where the approximation quality is measured with respect to the standard Gaussian distribution $N(0,I_n)$ and the complexity of an…
This work presents an entropy stable discontinuous Galerkin (DG) spectral element approximation for systems of non-linear conservation laws with general geometric (h) and polynomial order (p) non-conforming rectangular meshes. The crux of…
A well-known result across information theory, machine learning, and statistical physics shows that the maximum entropy distribution under a mean constraint has an exponential form called the Gibbs-Boltzmann distribution. This is used for…