Related papers: Samplers and Extractors for Unbounded Functions
We present a new lower bound on the differential entropy rate of stationary processes whose sequences of probability density functions fulfill certain regularity conditions. This bound is obtained by showing that the gap between the…
We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for…
We study universal approximation of continuous functionals on compact subsets of products of Hilbert spaces. We prove that any such functional can be uniformly approximated by models that first take finitely many continuous linear…
Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent…
We study the Kullback--Leibler (KL) divergence approximation theory of Gaussian mixture models (GMMs) by isolating an abstract mechanism behind several necessary-and-sufficient statements. The necessity direction is universal: if a density…
Recent research has revealed that deep generative models including flow-based models and Variational Autoencoders may assign higher likelihoods to out-of-distribution (OOD) data than in-distribution (ID) data. However, we cannot sample OOD…
This work studies finite sample approximations of the exact and entropic regularized Wasserstein distances between centered Gaussian processes and, more generally, covariance operators of functional random processes. We first show that…
Carbery proved that if $u:\mathbb{R}^n \rightarrow \mathbb{R}$ is a positive, strictly convex function satisfying $\det D^2u \geq 1$, then we have the estimate $$ \left| \left\{x \in \mathbb{R}^n: u(x) \leq s \right\} \right| \lesssim_n…
The goal in thinning is to summarize a dataset using a small set of representative points. Remarkably, sub-Gaussian thinning algorithms like Kernel Halving and Compress can match the quality of uniform subsampling while substantially…
We study the recovery of multivariate functions from reproducing kernel Hilbert spaces in the uniform norm. Our main interest is to obtain preasymptotic estimates for the corresponding sampling numbers. We obtain results in terms of the…
We consider minimum variance estimation within the sparse linear Gaussian model (SLGM). A sparse vector is to be estimated from a linearly transformed version embedded in Gaussian noise. Our analysis is based on the theory of reproducing…
Traditional measures of smoothness often fail to provide accurate $L_p$-error estimates for approximation by sampling or interpolation operators, especially for functions with low smoothness. To address this issue, we introduce a modified…
Let $\tau_k$ be the $k$-fold divisor function. By constructing an approximant of $\tau_k$, denoted as $\tau_k^*$, which is a normalized truncation of the $k$-fold divisor function, we prove that when $\exp\left(C\log^{1/2}X(\log\log…
We consider the problem of estimating small ball probabilities $\mathbb P\{f(G) \leqslant \delta \mathbb Ef(G)\}$ for sub-additive,positively homogeneous functions $f$ with respect to the Gaussian measure. We establish estimates that depend…
This paper addresses the following question: given a sample of i.i.d. random variables with finite variance, can one construct an estimator of the unknown mean that performs nearly as well as if the data were normally distributed? One of…
We study maximal averages associated with singular measures on $\rr$. Our main result is a construction of singular Cantor-type measures supported on sets of Hausdorff dimension $1 - \epsilon$, $0 \leq \epsilon < {1/3}$ for which the…
The families of $f$-divergences (e.g. the Kullback-Leibler divergence) and Integral Probability Metrics (e.g. total variation distance or maximum mean discrepancies) are widely used to quantify the similarity between probability…
Model averaging is a useful and robust method for dealing with model uncertainty in statistical analysis. Often, it is useful to consider data subset selection at the same time, in which model selection criteria are used to compare models…
Good robust estimators can be tuned to combine a high breakdown point and a specified asymptotic efficiency at a central model. This happens in regression with MM- and tau-estimators among others. However, the finite-sample efficiency of…
We develop techniques for determining an explicit Berry-Esseen bound in the Kolmogorov distance for the normal approximation of a ratio of Gaussian functionals. We provide an upper bound in terms of the third and fourth cumulants, using…