Related papers: Concentration bounds for intrinsic dimension estim…

We consider the classification problem of a high-dimensional mixture of two Gaussians with general covariance matrices. Using the replica method from statistical physics, we investigate the asymptotic behavior of a general class of…

This paper addresses distributed learning of a complex object for multiple networked robots based on distributed optimization and kernel-based support vector machine. In order to overcome a fundamental limitation of polynomial kernels…

We provide a sharp quantitative version of the Gaussian concentration inequality: for every $r>0$, the difference between the measure of the $r$-enlargement of a given set and the $r$-enlargement of a half-space controls the square of the…

We present a new method for estimating the frontier of a multidimensional sample. The estimator is based on a kernel regression on the power-transformed data. We assume that the exponent of the transformation goes to infinity while the…

In the this paper, the authors propose to estimate the density of a targeted population with a weighted kernel density estimator (wKDE) based on a weighted sample. Bandwidth selection for wKDE is discussed. Three mean integrated squared…

Contemporary deep neural networks exhibit impressive results on practical problems. These networks generalize well although their inherent capacity may extend significantly beyond the number of training examples. We analyze this behavior in…

Recent outer bounds on the capacity region of Gaussian interference channels are generalized to $m$-user channels with $m>2$ and asymmetric powers and crosstalk coefficients. The bounds are again shown to give the sum-rate capacity for…

The phenomenon of entropy concentration provides strong support for the maximum entropy method, MaxEnt, for inferring a probability vector from information in the form of constraints. Here we extend this phenomenon, in a discrete setting,…

We study the distribution of a fully connected neural network with random Gaussian weights and biases in which the hidden layer widths are proportional to a large constant $n$. Under mild assumptions on the non-linearity, we obtain…

The paper considers probability distribution, density, conditional distribution and density and conditional moments as well as their kernel estimators in spaces of generalized functions. This approach does not require restrictions on…

We consider the overfitting behavior of minimum norm interpolating solutions of Gaussian kernel ridge regression (i.e. kernel ridgeless regression), when the bandwidth or input dimension varies with the sample size. For fixed dimensions, we…

The problem of establishing out-of-sample bounds for the values of an unkonwn ground-truth function is considered. Kernels and their associated Hilbert spaces are the main formalism employed herein along with an observational model where…

Estimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data. The ability to better predict GE based on a single training set may yield overarching DNN…

One reason why standard formulations of the central limit theorems are not applicable in high-dimensional and non-stationary regimes is the lack of a suitable limit object. Instead, suitable distributional approximations can be used, where…

The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…

To better understand the theoretical behavior of large neural networks, several works have analyzed the case where a network's width tends to infinity. In this regime, the effect of random initialization and the process of training a neural…

Local polynomial density (LPD) estimators are widely used for inference on boundary features of the density function. Contrary to conventional wisdom, we show that kernel choice substantially affects efficiency. Theory, simulations, and…

This paper derives a new strong Gaussian approximation bound for the sum of independent random vectors. The approach relies on the optimal transport theory and yields \textit{explicit} dependence on the dimension size $p$ and the sample…

Spectral properties of Gram matrices are central to high dimensional asymptotic analyses of statistical estimators in regression and covariance estimation. These properties, in turn, depend critically on the extreme singular values and…

This manuscript studies the Gaussian approximation of the coordinate-wise maximum of self-normalized statistics in high-dimensional settings. We derive an explicit Berry-Esseen bound under weak assumptions on the absolute moments. When the…