Related papers: Numerical evaluation of Gaussian mixture entropy

Estimators derived from a divergence criterion such as $\varphi-$divergences are generally more robust than the maximum likelihood ones. We are interested in particular in the so-called MD$\varphi$DE, an estimator built using a dual…

We study the approximation of expectations $\E(f(X))$ for Gaussian random elements $X$ with values in a separable Hilbert space $H$ and Lipschitz continuous functionals $f \colon H \to \R$. We consider restricted Monte Carlo algorithms,…

The classical Gaussian ensembles of random matrices can be constructed by maximizing Boltzmann-Gibbs-Shannon's entropy, S_{BGS} = - \int d{\bf H} [P({\bf H})] \ln [P({\bf H})], with suitable constraints. Here we construct and analyze…

Near-Gaussian probability densities are common in many important physical applications. Here we develop an asymptotic expansion methodology for computing entropic functionals for such densities. The expansion proposed is a close relative of…

We use entropy numbers in combination with the polynomial method to derive a new general lower bound for the n-th minimal error in the quantum setting of information-based complexity. As an application, we improve some lower bounds on…

We describe a general technique that yields the first {\em Statistical Query lower bounds} for a range of fundamental high-dimensional learning problems involving Gaussian distributions. Our main results are for the problems of (1) learning…

We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…

The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs be obtained from the data sample itself. We propose a semi-parametric…

We provide another approach to Friedland's result that the topological entropy $h$ of a symmetric nearest-neighbor subshift is computable. Instead of the previous algebraic technique, our approach is mostly combinatorial and involves only…

We revisit entropic formulations of the uncertainty principle for an arbitrary pair of positive operator-valued measures (POVM) $A$ and $B$, acting on finite dimensional Hilbert space. Salicr\'u generalized $(h,\phi)$-entropies, including…

Tsallis' non-extensive entropy is extended to incorporate the dependence on affinities between the microstates of a system. At the core of our construction of the extended entropy ($\mathcal{H}$) is the concept of the effective number of…

We present a simple combinatorial framework for establishing approximate tensorization of variance and entropy in the setting of spin systems (a.k.a. undirected graphical models) based on balanced separators of the underlying graph. Such…

The Poisson model is frequently employed to describe count data, but in a Bayesian context it leads to an analytically intractable posterior probability distribution. In this work, we analyze a variational Gaussian approximation to the…

In this paper we study algorithms to find a Gaussian approximation to a target measure defined on a Hilbert space of functions; the target measure itself is defined via its density with respect to a reference Gaussian measure. We employ the…

Given $m \ge 2$ discrete probability distributions over $n$ states each, the minimum-entropy coupling is the minimum-entropy joint distribution whose marginals are the same as the input distributions. Computing the minimum-entropy coupling…

We prove bounds on statistical distances between high-dimensional exchangeable mixture distributions (which we call \emph{permutation mixtures}) and their i.i.d. counterparts. Our results are based on a novel method for controlling $\chi^2$…

Hybrid Bayesian Networks (HBNs), which contain both discrete and continuous variables, arise naturally in many application areas (e.g., image understanding, data fusion, medical diagnosis, fraud detection). This paper concerns inference in…

This paper first focuses on deriving an alternative approach for proving an extremal entropy inequality (EEI), originally presented in [11]. The proposed approach does not rely on the channel enhancement technique, and has the advantage…

Variational inference (VI) is a popular approach in Bayesian inference, that looks for the best approximation of the posterior distribution within a parametric family, minimizing a loss that is typically the (reverse) Kullback-Leibler (KL)…

We introduce a subexponential algorithm for geometric solving of multivariate polynomial equation systems whose bit complexity depends mainly on intrinsic geometric invariants of the solution set. From this algorithm, we derive a new…