Related papers: MMSE Bounds Under Kullback-Leibler Divergence Cons…
In this paper, we compare the performance of two methods for estimating Bayesian networks from data containing exogenous variables and random effects. The first method is fully Bayesian in which a prior distribution is placed on the…
We prove minimax bounds for estimating Gaussian location mixtures on $\mathbb{R}^d$ under the squared $L^2$ and the squared Hellinger loss functions. Under the squared $L^2$ loss, we prove that the minimax rate is upper and lower bounded by…
This work presents joint minimum mean-square error (MMSE) consensus algorithm and relay selection algorithms for distributed beamforming. We propose joint MMSE consensus relay and selection schemes with a total power constraint and local…
This paper introduces a variational approximation framework using direct optimization of what is known as the {\it scale invariant Alpha-Beta divergence} (sAB divergence). This new objective encompasses most variational objectives that use…
Discriminator Guidance has become a popular method for efficiently refining pre-trained Score-Matching Diffusion models. However, in this paper, we demonstrate that the standard implementation of this technique does not necessarily lead to…
Mean embeddings provide an extremely flexible and powerful tool in machine learning and statistics to represent probability distributions and define a semi-metric (MMD, maximum mean discrepancy; also called N-distance or energy distance),…
Score-matching generative models have proven successful at sampling from complex high-dimensional data distributions. In many applications, this distribution is believed to concentrate on a much lower $d$-dimensional manifold embedded into…
Consider the problem of estimating the mean of a Gaussian random vector when the mean vector is assumed to be in a given convex set. The most natural solution is to take the Euclidean projection of the data vector on to this convex set; in…
Minimax lower bounds are pessimistic in nature: for any given estimator, minimax lower bounds yield the existence of a worst-case target vector $\beta^*_{worst}$ for which the prediction error of the given estimator is bounded from below.…
In many estimation theory and statistical analysis problems, the true data model is unknown, or partially unknown. To describe the model generating the data, parameterized models of some degree are used. A question that arises is which…
This paper extends the standard chaining technique to prove excess risk upper bounds for empirical risk minimization with random design settings even if the magnitude of the noise and the estimates is unbounded. The bound applies to many…
We study concentration inequalities for the Kullback--Leibler (KL) divergence between the empirical distribution and the true distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes…
The I-MMSE formula connects two important quantities in information theory and estimation theory: the mutual information and the minimum mean-squared error (MMSE). It states that in a scalar Gaussian channel, the derivative of the mutual…
In this paper the Gaussian quasi maximum likelihood estimator (GQMLE) is generalized by applying a transform to the probability distribution of the data. The proposed estimator, called measure-transformed GQMLE (MT-GQMLE), minimizes the…
We study the problem of $K$-armed bandits with reward distributions belonging to a one-parameter exponential distribution family. In the literature, several criteria have been proposed to evaluate the performance of such algorithms,…
We study the problem of estimating a distribution over a finite alphabet from an i.i.d. sample, with accuracy measured in relative entropy (Kullback-Leibler divergence). While optimal bounds on the expected risk are known, high-probability…
We study frequentist risk properties of predictive density estimators for mean mixtures of multivariate normal distributions, involving an unknown location parameter $\theta \in \mathbb{R}^d$, and which include multivariate skew normal…
We present a unified technique for sequential estimation of convex divergences between distributions, including integral probability metrics like the kernel maximum mean discrepancy, $\varphi$-divergences like the Kullback-Leibler…
This paper establishes quantitative correlation inequalities between monotone events and structured threshold objects in both the discrete cube and Gaussian space. We prove that for any increasing balanced family, there exists a linear…
When recovering a sparse signal from noisy compressive linear measurements, the distribution of the signal's non-zero coefficients can have a profound effect on recovery mean-squared error (MSE). If this distribution was apriori known, then…