Related papers: A Generalized Notion of Completeness and Its Appli…
The necessary information for specifying a complex system may not be completely accessible to us, i.e., to mathematical treatments. This is not to be confounded with the incompleteness of our knowledge about whatever systems or nature,…
This paper develops the sufficiency principle suitable for data reduction in decentralized inference systems. Both parallel and tandem networks are studied and we focus on the cases where observations at decentralized nodes are…
The matrix completion problem consists in reconstructing a matrix from a sample of entries, possibly observed with noise. A popular class of estimator, known as nuclear norm penalized estimators, are based on minimizing the sum of a data…
Improvement guarantees for semi-supervised classifiers can currently only be given under restrictive conditions on the data. We propose a general way to perform semi-supervised parameter estimation for likelihood-based classifiers for…
We study the problem of estimating a distribution over a finite alphabet from an i.i.d. sample, with accuracy measured in relative entropy (Kullback-Leibler divergence). While optimal bounds on the expected risk are known, high-probability…
The advent of data science has spurred interest in estimating properties of distributions over large alphabets. Fundamental symmetric properties such as support size, support coverage, entropy, and proximity to uniformity, received most…
Smooth Estimation of probability density and distribution functions from its sample is an attractive and an important problem that has applications in several fields such as, business, medicine, and environment. This article introduces a…
Equilibrium statistical physics is considered from the point of view of statistical estimation theory. This involves the notions of statistical model, of estimators, and of exponential family. A useful property of the latter is the…
A simple characterization of uniformly minimum variance unbiased estimators (UMVUEs) is provided (in the case when the sample space is finite) in terms of a linear independence condition on the likelihood functions corresponding to the…
Popular deterministic approximations of posterior distributions from, e.g. the Laplace method, variational Bayes and expectation-propagation, generally rely on symmetric approximating families, often taken to be Gaussian. This choice…
Uncertainty estimation aims to evaluate the confidence of a trained deep neural network. However, existing uncertainty estimation approaches rely on low-dimensional distributional assumptions and thus suffer from the high dimensionality of…
The network data has attracted considerable attention in modern statistics. In research on complex network data, one key issue is finding its underlying connection structure given a network sample. The methods that have been proposed in…
Sparse representation over redundant dictionaries constitutes a good model for many classes of signals (e.g., patches of natural images, segments of speech signals, etc.). However, despite its popularity, very little is known about the…
A famous characterization theorem due to C.F. Gauss states that the maximum likelihood estimator (MLE) of the parameter in a location family is the sample mean for all samples of all sample sizes if and only if the family is Gaussian. There…
Information-theoretic measures have been widely adopted in the design of features for learning and decision problems. Inspired by this, we look at the relationship between i) a weak form of information loss in the Shannon sense and ii) the…
One of the main concepts in quantum physics is a density matrix, which is a symmetric positive definite matrix of trace one. Finite probability distributions can be seen as a special case when the density matrix is restricted to be…
The {\lambda}-exponential family has recently been proposed to generalize the exponential family. While the exponential family is well-understood and widely used, this it not the case of the {\lambda}-exponential family. However, many…
We consider the minimization of the number of non-zero coefficients (the $\ell_0$ "norm") of the representation of a data set in terms of a dictionary under a fidelity constraint. (Both the dictionary and the norm defining the constraint…
We introduce estimation and test procedures through divergence minimiza- tion for models satisfying linear constraints with unknown parameter. These procedures extend the empirical likelihood (EL) method and share common features with…
As of 2005, sampling has been incorporated in all major database systems. While efficient sampling techniques are realizable, determining the accuracy of an estimate obtained from the sample is still an unresolved problem. In this paper, we…