Related papers: Kullback-Leibler Approximation for Probability Mea…
We address the problem of continual learning in multi-task Gaussian process (GP) models for handling sequential input-output observations. Our approach extends the existing prior-posterior recursion of online Bayesian inference, i.e.\ past…
We derive the class of covariant measurements which are optimal according to the maximum likelihood criterion. The optimization problem is fully resolved in the case of pure input states, under the physically meaningful hypotheses of…
To achieve scalable and accurate inference for latent Gaussian processes, we propose a variational approximation based on a family of Gaussian distributions whose covariance matrices have sparse inverse Cholesky (SIC) factors. We combine…
In the present paper and the companion paper [8] a probabilistic (statistical mechanical) approach to the study of canonical metrics and measures on a complex algebraic variety X is introduced. On any such variety with positive Kodaira…
We consider the asymptotic behavior of posterior distributions if the model is misspecified. Given a prior distribution and a random sample from a distribution $P_0$, which may not be in the support of the prior, we show that the posterior…
Optimization in the Bures-Wasserstein space has been gaining popularity in the machine learning community since it draws connections between variational inference and Wasserstein gradient flows. The variational inference objective function…
We describe a construction process of a relevant measure in any non-empty compact metric space. This probability measure has invariance properties with respect to isometric maps defined on open sets. These properties imply that this measure…
Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering. This task may be cast as an optimization problem over all probability measures, and an initial…
Obtaining an accurate estimate of the underlying covariance matrix from finite sample size data is challenging due to sample size noise. In recent years, sophisticated covariance-cleaning techniques based on random matrix theory have been…
We show that the Kullback-Leibler distance is a good measure of the statistical uncertainty of correlation matrices estimated by using a finite set of data. For correlation matrices of multivariate Gaussian variables we analytically…
Modern applications of Bayesian inference involve models that are sufficiently complex that the corresponding posterior distributions are intractable and must be approximated. The most common approximation is based on Markov chain Monte…
Variational Inference (VI) is a popular alternative to asymptotically exact sampling in Bayesian inference. Its main workhorse is optimization over a reverse Kullback-Leibler divergence (RKL), which typically underestimates the tail of the…
Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably…
Divergences are quantities that measure discrepancy between two probability distributions and play an important role in various fields such as statistics and machine learning. Divergences are non-negative and are equal to zero if and only…
Dimension reduction is a technique used to transform data from a high-dimensional space into a lower-dimensional space, aiming to retain as much of the original information as possible. This approach is crucial in many disciplines like…
Common statistical measures of uncertainty such as $p$-values and confidence intervals quantify the uncertainty due to sampling, that is, the uncertainty due to not observing the full population. However, sampling is not the only source of…
A method for selecting a graphical model for $p$-vector-valued stationary Gaussian time series was recently proposed by Matsuda and uses the Kullback-Leibler divergence measure to define a test statistic. This statistic was used in a…
The accurate asymptotic evaluation of marginal likelihood integrals is a fundamental problem in Bayesian statistics. Following the approach introduced by Watanabe, we translate this into a problem of computational algebraic geometry,…
We consider finite-dimensional linear Gaussian Bayesian inverse problems with uncorrelated sensor measurements. In this setting, it is known that the expected information gain, quantified by the expected Kullback-Leibler divergence from the…
We study the problem of closeness testing for continuous distributions and its implications for causal discovery. Specifically, we analyze the sample complexity of distinguishing whether two multidimensional continuous distributions are…