Related papers: A Quantitative Occam's Razor
Parameter ensembles or sets of random effects constitute one of the cornerstones of modern statistical practice. This is especially the case in Bayesian hierarchical models, where several decision theoretic frameworks can be deployed. The…
Bayesian inference provides a uniquely rigorous approach to obtain principled justification for uncertainty in predictions, yet it is difficult to articulate suitably general prior belief in the machine learning context, where computational…
This paper concerns the robust regression model when the number of predictors and the number of observations grow in a similar rate. Theory for M-estimators in this regime has been recently developed by several authors [El Karoui et al.,…
Meta-analysis is a statistical method used in evidence synthesis for combining, analyzing and summarizing studies that have the same target endpoint and aims to derive a pooled quantitative estimate using fixed and random effects models or…
I show that the maximum entropy principle can be replaced by a more natural assumption, that there exists a phenomenological function of entropy consistent with the microscopic model. The requirement of existence provides then a unique…
Inferring the value of a property of a large stochastic system is a difficult task when the number of samples is insufficient to reliably estimate the probability distribution. The Bayesian estimator of the property of interest requires the…
I define a natural measure of the complexity of a parametric distribution relative to a given true distribution called the {\it razor} of a model family. The Minimum Description Length principle (MDL) and Bayesian inference are shown to…
We propose a new method for conducting Bayesian prediction that delivers accurate predictions without correctly specifying the unknown true data generating process. A prior is defined over a class of plausible predictive models. After…
The entropy is one of the most applicable uncertainty measures in many statistical and en- gineering problems. In statistical literature, the entropy is used in calculation of the Kullback- Leibler (KL) information which is a powerful mean…
This is a tutorial aimed at illustrating some recent developments in quantum parameter estimation beyond the Cram\`er-Rao bound, as well as their applications in quantum metrology. Our starting point is the observation that there are…
The ranking problem is to order a collection of units by some unobserved parameter, based on observations from the associated distribution. This problem arises naturally in a number of contexts, such as business, where we may want to rank…
We explore the interplay between random and deterministic phenomena using a representation of uncertainty based on the measure-theoretic concept of outer measure. The meaning of the analogues of different probabilistic concepts is…
This paper introduces a method for efficiently inferring a high-dimensional distributed quantity from a few observations. The quantity of interest (QoI) is approximated in a basis (dictionary) learned from a training set. The coefficients…
Model inadequacy and measurement uncertainty are two of the most confounding aspects of inference and prediction in quantitative sciences. The process of scientific inference (the inverse problem) and prediction (the forward problem)…
Three different inferential problems related to a two dimensional categorical data from a Bayesian perspective have been discussed in this article. Conjugate prior distribution with symmetric and asymmetric hyper parameters are considered.…
While there has been progress in developing non-vacuous generalization bounds for deep neural networks, these bounds tend to be uninformative about why deep learning works. In this paper, we develop a compression approach based on…
We derive an asymptotic lower bound on the Bayes risk when N identical quantum systems whose state depends on a vector of unknown parameters are jointly measured in an arbitrary way and the parameters of interest estimated on the basis of…
Using the convex structure of positive operator value measurements and of several quantities used in quantum metrology, such as quantum Fisher information or the quantum Van Trees information, we present an efficient numerical method to…
Quantum estimation theory provides optimal observations for various estimation problems for unknown parameters in the state of the system under investigation. However, the theory has been developed under the assumption that every observable…
The exact statistics of an arbitrary quantum observable is analytically obtained. Due to the probabilistic nature of a sequence of intermediate measurements and stochastic fluctuations induced by the interaction with the environment, the…