Related papers: Occam's Ghost
We introduce the concept of compressed convolution, a technique to convolve a given data set with a large number of non-orthogonal kernels. In typical applications our technique drastically reduces the effective number of computations. The…
Probabilistic Regression refers to predicting a full probability density function for the target conditional on the features. We present a nonparametric approach to this problem which combines base classifiers (typically gradient boosted…
The Waxman random graph is a generalisation of the simple Erd\H{o}s-R\'enyi or Gilbert random graph. It is useful for modelling physical networks where the increased cost of longer links means they are less likely to be built, and thus less…
As in other estimation scenarios, likelihood based estimation in the normal mixture set-up is highly non-robust against model misspecification and presence of outliers (apart from being an ill-posed optimization problem). A robust…
Detecting symmetry from data is a fundamental problem in signal analysis, providing insight into underlying structure and constraints. When data emerge as trajectories of dynamical systems, symmetries encode structural properties of the…
We study the relationship between gradient-based optimization of parametric models (e.g., neural networks) and optimization of linear combinations of random features. Our main result shows that if a parametric model can be learned using…
In a circular deconvolution model we consider the fully data driven density estimation of a circular random variable where the density of the additive independent measurement error is unknown. We have at hand two independent iid samples,…
Histograms are convenient non-parametric density estimators, which continue to be used ubiquitously. Summary quantities estimated from histogram-based probability density models depend on the choice of the number of bins. We introduce a…
Roughly, a metric space has padding parameter $\beta$ if for every $\Delta>0$, there is a stochastic decomposition of the metric points into clusters of diameter at most $\Delta$ such that every ball of radius $\gamma\Delta$ is contained in…
We consider a Bayesian approach to model selection in Gaussian linear regression, where the number of predictors might be much larger than the number of observations. From a frequentist view, the proposed procedure results in the penalized…
Subsets of F_2^n that are eps-biased, meaning that the parity of any set of bits is even or odd with probability eps close to 1/2, are powerful tools for derandomization. A simple randomized construction shows that such sets exist of size…
The problem we concentrate on is as follows: given (1) a convex compact set $X$ in ${\mathbb{R}}^n$, an affine mapping $x\mapsto A(x)$, a parametric family $\{p_{\mu}(\cdot)\}$ of probability densities and (2) $N$ i.i.d. observations of the…
High throughput genetic sequencing arrays with thousands of measurements per sample and a great amount of related censored clinical data have increased demanding need for better measurement specific model selection. In this paper we…
In this paper, we will discuss how to generalize nonparametric density estimators to MLE parametric estimators. Basing on the Parzen window theory and using the advantages of probability amplitude of quantum theory, we model a nonlinear…
Modern neural network architectures typically have many millions of parameters and can be pruned significantly without substantial loss in effectiveness which demonstrates they are over-parameterized. The contribution of this work is…
In a previous article, a least square regression estimation procedure was proposed: first, we condiser a family of functions and study the properties of an estimator in every unidimensionnal model defined by one of these functions; we then…
The inversion of electromagnetic induction data to a conductivity profile is an ill-posed problem. Regularization improves the stability of the inversion and, based on Occam's razor principle, a smoothing constraint is typically used.…
This paper presents minimax rates for density estimation when the data dimension $d$ is allowed to grow with the number of observations $n$ rather than remaining fixed as in previous analyses. We prove a non-asymptotic lower bound which…
It is a common contention that it is an ``impossible mission'' to exactly determine the minimum sample size for the estimation of a binomial parameter with prescribed margin of error and confidence level. In this paper, we investigate such…
The Minimum Description Length (MDL) principle offers a formal framework for applying Occam's razor in machine learning. However, its application to neural networks such as Transformers is challenging due to the lack of a principled,…