Related papers: Universality Laws for High-Dimensional Learning wi…
Kernel methods augmented with random features give scalable algorithms for learning from big data. But it has been computationally hard to sample random features according to a probability distribution that is optimized for the data, so as…
Continual learning, the ability of a model to adapt to an ongoing sequence of tasks without forgetting earlier ones, is a central goal of artificial intelligence. To better understand its underlying mechanisms, we study the limitations of…
Despite the fact that generative models are extremely successful in practice, the theory underlying this phenomenon is only starting to catch up with practice. In this work we address the question of the universality of generative models:…
In a general class of Bayesian nonparametric models, we prove that the posterior distribution can be asymptotically approximated by a Gaussian process. Our results apply to nonparametric exponential family that contains both Gaussian and…
We develop a uniform inference theory for high-dimensional slope parameters in threshold regression models, allowing for either cross-sectional or time series data. We first establish oracle inequalities for prediction errors, and L1…
We give a constructive proof for the superbosonization formula for invariant random matrix ensembles, which is the supersymmetry analog of the theory of Wishart matrices. Formulas are given for unitary, orthogonal and symplectic symmetry,…
We study the problem of learning mixtures of $k$ Gaussians in $d$ dimensions. We make no separation assumptions on the underlying mixture components: we only require that the covariance matrices have bounded condition number and that the…
We prove a general multidimensional invariance principle for a family of U-statistics based on freely independent non-commutative random variables of the type $U_n(S)$, where $U_n(x)$ is the $n$-th Chebyshev polynomial and $S$ is a standard…
A general theory for Gaussian mean estimation that automatically adapts to unknown sparsity under arbitrary norms is proposed. The theory is applied to produce adaptively minimax rate-optimal estimators in high dimensional regression and…
Humans rely on effective representations to learn from few examples and abstract useful information from sensory data. Inducing such representations in machine learning models has been shown to improve their performance on various…
We study uniform consistency in nonparametric mixture models as well as closely related mixture of regression (also known as mixed regression) models, where the regression functions are allowed to be nonparametric and the error…
Turing machines and spin models share a notion of universality according to which some simulate all others. Is there a theory of universality that captures this notion? We set up a categorical framework for universality which includes as…
One of the major themes of random matrix theory is that many asymptotic properties of traditionally studied distributions of random matrices are universal. We probe the edges of universality by studying the spectral properties of random…
Bayesian learning is built on an assumption that the model space contains a true reflection of the data generating mechanism. This assumption is problematic, particularly in complex data environments. Here we present a Bayesian…
Shortened abstract: Given a constrained minimization problem, under what conditions does there exist a related, unconstrained problem having the same minimum points? This basic question in global optimization motivates this paper, which…
We introduce a new approach to probabilistic unsupervised learning based on the recognition-parametrised model (RPM): a normalised semi-parametric hypothesis class for joint distributions over observed and latent variables. Under the key…
With the help of Generalized Estimating Equations, we identify locally D-optimal crossover designs for generalized linear models. We adopt the variance of parameters of interest as the objective function, which is minimized using…
We consider two classical ensembles of the random matrix theory: the Wigner matrices and sample covariance matrices, and prove Central Limit Theorem for linear eigenvalue statistics under rather weak (comparing with results known before)…
This paper develops a general approach to characterize the long-time trajectory behavior of nonconvex gradient descent in generalized single-index models in the large aspect ratio regime. In this regime, we show that for each iteration the…
We introduce parametrisation of that property of the available training dataset, that necessitates an inhomogeneous correlation structure for the function that is learnt as a model of the relationship between the pair of variables,…