Related papers: Nearest neighbor density functional estimation fro…
We provide finite-sample analysis of a general framework for using k-nearest neighbor statistics to estimate functionals of a nonparametric continuous probability density, including entropies and divergences. Rather than plugging a…
Estimating expected polynomials of density functions from samples is a basic problem with numerous applications in statistics and information theory. Although kernel density estimators are widely used in practice for such functional…
Statistical applications often involve the calculation of intractable multidimensional integrals. The Laplace formula is widely used to approximate such integrals. However, in high-dimensional or small sample size problems, the shape of the…
A non-parametric k-nearest neighbour based entropy estimator is proposed. It improves on the classical Kozachenko-Leonenko estimator by considering non-uniform probability densities in the region of k-nearest neighbours around each sample…
We consider nonparametric estimation of $L_2$, Renyi-$\alpha$ and Tsallis-$\alpha$ divergences between continuous distributions. Our approach is to construct estimators for particular integral functionals of two densities and translate them…
We consider density estimators based on the nearest neighbors method applied to discrete point distibutions in spaces of arbitrary dimensionality. If the density is constant, the volume of a hypersphere centered at a random location is…
The problem of accurate nonparametric estimation of distributional functionals (integral functionals of one or more probability distributions) has received recent interest due to their wide applicability in signal processing, information…
Biased sampling and missing data complicates statistical problems ranging from causal inference to reinforcement learning. We often correct for biased sampling of summary statistics with matching methods and importance weighting. In this…
Nearest neighbor (NN) matching as a tool to align data sampled from different groups is both conceptually natural and practically well-used. In a landmark paper, Abadie and Imbens (2006) provided the first large-sample analysis of NN…
Expected values weighted by the inverse of a multivariate density or, equivalently, Lebesgue integrals of regression functions with multivariate regressors occur in various areas of applications, including estimating average treatment…
Laplace's method approximates a target density with a Gaussian distribution at its mode. It is computationally efficient and asymptotically exact for Bayesian inference due to the Bernstein-von Mises theorem, but for complex targets and…
In a previous article, a least square regression estimation procedure was proposed: first, we condiser a family of functions and study the properties of an estimator in every unidimensionnal model defined by one of these functions; we then…
The method of maximum entropy has proven to be a rather powerful way to solve the inverse problem consisting of determining a probability density $f_S(s)$ on $[0,\infty)$ from the knowledge of the expected value of a few generalized…
Estimating mutual information from i.i.d. samples drawn from an unknown joint density function is a basic statistical problem of broad interest with multitudinous applications. The most popular estimator is one proposed by Kraskov and…
This paper introduces a class of k-nearest neighbor ($k$-NN) estimators called bipartite plug-in (BPI) estimators for estimating integrals of non-linear functions of a probability density, such as Shannon entropy and R\'enyi entropy. The…
The Laplace approximation is an old, but frequently used method to approximate integrals for Bayesian calculations. In this paper we develop an extension of the Laplace approximation, by applying it iteratively to the residual, i.e., the…
Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor…
From a suitable integral representation of the Laplace transform of a positive semi-definite quadratic form of independent real random variables with not necessarily identical densities a univariate integral representation is derived for…
Many models require integrals of high-dimensional functions: for instance, to obtain marginal likelihoods. Such integrals may be intractable, or too expensive to compute numerically. Instead, we can use the Laplace approximation (LA). The…
We consider a generalization of the classifier-based density-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. The problem with most loss functions used for this task is that they implicitly…