Related papers: A Cram\'er-Rao inequality for non differentiable m…
We derive an unbiased estimator for expectations over discrete random variables based on sampling without replacement, which reduces variance as it avoids duplicate samples. We show that our estimator can be derived as the…
This paper presents a Cramer-Rao bound (CRB) for the estimation of parameters confined to an arbitrary set. Unlike existing results that rely on equality or inequality constraints, manifold structures, or the nonsingularity of the Fisher…
The variance and the entropy power of a continuous random variable are bounded from below by the reciprocal of its Fisher information through the Cram\'{e}r-Rao bound and the Stam's inequality respectively. In this note, we introduce the…
In many practical parameter estimation problems, such as coefficient estimation of polynomial regression, the true model is unknown and thus, a model selection step is performed prior to estimation. The data-based model selection step…
The widely used quantum Cramer-Rao bound (QCRB) sets a lower bound for the mean square error of unbiased estimators in quantum parameter estimation, however, in general QCRB is only tight in the asymptotical limit. With a limited number of…
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the…
Recently, a Wasserstein analogue of the Cramer--Rao inequality has been developed using the Wasserstein information matrix (Otto metric). This inequality provides a lower bound on the Wasserstein variance of an estimator, which quantifies…
A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein--Weiss family of bounds for the mean squared error and relies…
In this paper, we address the theoretical limitations in reconstructing sparse signals (in a known complete basis) using compressed sensing framework. We also divide the CS to non-blind and blind cases. Then, we compute the Bayesian…
We develop a maximum-likelihood based method for regression in a setting where the dependent variable is a random graph and covariates are available on a graph-level. The model generalizes the well-known $\beta$-model for random graphs by…
Recent progress in deep latent variable models has largely been driven by the development of flexible and scalable variational inference methods. Variational training of this type involves maximizing a lower bound on the log-likelihood,…
Many statistical estimators are defined as the fixed point of a data-dependent operator, with estimators based on minimizing a cost function being an important special case. The limiting performance of such estimators depends on the…
This work presents a geometric refinement of the classical Cram\'er--Rao bound (CRB) in the non-asymptotic regime by incorporating curvature-aware corrections based on the second fundamental form associated with the statistical model…
We construct an unbiased estimator for function value evaluated at the solution of a partial differential equation with random coefficients. We show that the variance and expected computational cost of our estimator are finite and our…
Unbiased and consistent variance estimators generally do not exist for design-based treatment effect estimators because experimenters never observe more than one potential outcome for any unit. The problem is exacerbated by interference and…
Estimation under model misspecification arises in many signal processing problems, where the assumed observation model deviates from the true data-generating mechanism due to errors or simplifications. The misspecified Cram\'er-Rao bound…
Learning in models with discrete latent variables is challenging due to high variance gradient estimators. Generally, approaches have relied on control variates to reduce the variance of the REINFORCE estimator. Recent work (Jang et al.…
How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational…
When randomized ensembles such as bagging or random forests are used for binary classification, the prediction error of the ensemble tends to decrease and stabilize as the number of classifiers increases. However, the precise relationship…
The constrained Cramer-Rao bound (CCRB) is a lower bound on the mean-squared-error (MSE) of estimators that satisfy some unbiasedness conditions. Although the CCRB unbiasedness conditions are satisfied asymptotically by the constrained…