Related papers: Theoretical Error Analysis of Entropy Approximatio…
We study the problem of quantifying how far an empirical distribution deviates from Gaussianity under the framework of optimal transport. By exploiting the cone geometry of the relative translation invariant quadratic Wasserstein space, we…
We consider the problem of Gaussian mixture clustering in the high-dimensional limit where the data consists of $m$ points in $n$ dimensions, $n,m \rightarrow \infty$ and $\alpha = m/n$ stays finite. Using exact but non-rigorous methods…
A well-known result across information theory, machine learning, and statistical physics shows that the maximum entropy distribution under a mean constraint has an exponential form called the Gibbs-Boltzmann distribution. This is used for…
The Expectation-Maximization (EM) algorithm is a fundamental tool in unsupervised machine learning. It is often used as an efficient way to solve Maximum Likelihood (ML) estimation problems, especially for models with latent variables. It…
The Bures-Hall distance metric between quantum states is a unique measure that satisfies various useful properties for quantum information processing. In this work, we study the statistical behavior of quantum entanglement over the…
Following [1], the aim of this paper is to analyze the relative weighted entropy involving the central moments weight functions. We compare the standard relative entropy with the weighted case in two particular forms of Gaussian…
A decision must often be made between heavy-tailed and Gaussian errors for a regression or a time series model, and the t-distribution is frequently used when it is assumed that the errors are heavy-tailed distributed. The performance of…
In this work, we study non-parametric estimation of joint probabilities of a given set of discrete and continuous random variables from their (empirically estimated) 2D marginals, under the assumption that the joint probability could be…
Uncertainty propagation in non-linear dynamical systems has become a key problem in various fields including control theory and machine learning. In this work we focus on discrete-time non-linear stochastic dynamical systems. We present a…
The effective sample size (ESS) measures the informational value of a probability distribution in terms of an equivalent number of study participants. The ESS plays a crucial role in estimating the Expected Value of Sample Information…
We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel…
We consider the problem of estimating means of two Gaussians in a 2-Gaussian mixture, which is not balanced and is corrupted by noise of an arbitrary distribution. We present a robust algorithm to estimate the parameters, together with…
Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond…
This paper investigates the approximation of Gaussian random variables in Banach spaces, focusing on the high-probability bounds for the approximation of Gaussian random variables using finitely many observations. We derive non-asymptotic…
We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…
Bayesian methods estimate a measure of uncertainty by using the posterior distribution. One source of difficulty in these methods is the computation of the normalizing constant. Calculating exact posterior is generally intractable and we…
In this study, a numerical quadrature for the generalized inverse Gaussian distribution is derived from the Gauss-Hermite quadrature by exploiting its relationship with the normal distribution. The proposed quadrature is not Gaussian, but…
The paper analyzes the entropy of a system composed by non-interacting and indistinguishable particles whose quantum state numbers are modelled as independent and identically distributed classical random variables. The crucial observation…
The question of polynomial learnability of probability distributions, particularly Gaussian mixture distributions, has recently received significant attention in theoretical computer science and machine learning. However, despite major…
The projected normal distribution, also known as the angular Gaussian distribution, is obtained by dividing a multivariate normal random variable $\mathbf{x}$ by its norm $\sqrt{\mathbf{x}^T \mathbf{x}}$. The resulting random variable…