Related papers: Numerical evaluation of Gaussian mixture entropy

This work studies the entropic regularization formulation of the 2-Wasserstein distance on an infinite-dimensional Hilbert space, in particular for the Gaussian setting. We first present the Minimum Mutual Information property, namely the…

We investigate the convergence properties of the EM algorithm when applied to overspecified Gaussian mixture models -- that is, when the number of components in the fitted model exceeds that of the true underlying distribution. Focusing on…

We address the estimation problem for general finite mixture models, with a particular focus on the elliptical mixture models (EMMs). Compared to the widely adopted Kullback-Leibler divergence, we show that the Wasserstein distance provides…

Given two discrete random variables $X$ and $Y,$ with probability distributions ${\bf p}=(p_1, \ldots , p_n)$ and ${\bf q}=(q_1, \ldots , q_m)$, respectively, denote by ${\cal C}({\bf p}, {\bf q})$ the set of all couplings of ${\bf p}$ and…

Many important physical problems, such as fluid structure interaction or conjugate heat transfer, require numerical methods that compute boundary derivatives or fluxes to high accuracy. This paper proposes a novel alternative to calculating…

Many methods for machine learning rely on approximate inference from intractable probability distributions. Variational inference approximates such distributions by tractable models that can be subsequently used for approximate inference.…

We introduce the Variational Holder (VH) bound as an alternative to Variational Bayes (VB) for approximate Bayesian inference. Unlike VB which typically involves maximization of a non-convex lower bound with respect to the variational…

We propose and analyze a structure-preserving approximation of the non-isothermal Cahn-Hilliard equation using conforming finite elements for the spatial discretization and a problem-specific mixed explicit-implicit approach for the…

Electronic structures are fully determined by the exchange-correlation (XC) potential. In this work, we develop a new method to construct reliable XC potentials by properly mixing the exact exchange and the local density approximation…

We propose a fast and scalable optimization method to solve chance or probabilistic constrained optimization problems governed by partial differential equations (PDEs) with high-dimensional random parameters. To address the critical…

We prove a proposition that the entropy of the system composed of finite $N$ molecules of ideal gas is the $q$-entropy or Havrda-Charv\'at-Tsallis entropy, which is also known as Tsallis entropy, with the entropic index…

We present an efficient algorithm to compute tight upper bounds of collision probability between two objects with positional uncertainties, whose error distributions are represented with non-Gaussian forms. Our approach can handle noisy…

This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…

As shown by M\'edard, the capacity of fading channels with imperfect channel-state information (CSI) can be lower-bounded by assuming a Gaussian channel input $X$ with power $P$ and by upper-bounding the conditional entropy $h(X|Y,\hat{H})$…

Many processes in chemistry and physics take place on timescales that cannot be explored using standard molecular dynamics simulations. This renders the use of enhanced sampling mandatory. Here we introduce an enhanced sampling method that…

The relative entropy between quantum states quantifies their distinguishability. The estimation of certain relative entropies has been investigated in the literature, e.g., the von Neumann relative entropy and sandwiched R\'enyi relative…

Gaussian and Laplacian entropy models are proved effective in learned point cloud attribute compression, as they assist in arithmetic coding of latents. However, we demonstrate through experiments that there is still unutilized information…

This work addresses the problem of state estimation in multivariable dynamic systems with quantized outputs, a common scenario in applications involving low-resolution sensors or communication constraints. A novel method is proposed to…

Gaussian mixtures are a powerful and widely used tool to model non-Gaussian estimation problems. They are able to describe measurement errors that follow arbitrary distributions and can represent ambiguity in assignment tasks like point set…

Purely multiplicative comparisons of quantum relative entropy are desirable but challenging to prove. We show such comparisons for relative entropies between comparable densities, including the relative entropy of a density with respect to…