Related papers: Mathematics of learning
We consider the logarithm of the characteristic polynomial of random permutation matrices, evaluated on a finite set of different points. The permutations are chosen with respect to the Ewens distribution on the symmetric group. We show…
We study the limiting behavior of smooth linear statistics of the spectrum of random permutation matrices in the mesoscopic regime, when the permutation follows one of the Ewens measures on the symmetric group. If we apply a smooth enough…
We consider the problem of learning function classes computed by neural networks with various activations (e.g. ReLU or Sigmoid), a task believed to be computationally intractable in the worst-case. A major open problem is to understand the…
This work is divided into three parts. The first part concerns polynomials in one variable with all real roots. We consider linear transformations that preserve real rootedness, as well as matrices that preserve interlacing. The second part…
The paper presents methods of eigenvalue localisation of regular matrix polynomials, in particular, stability of matrix polynomials is investigated. For this aim a stronger notion of hyperstability is introduced and widely discussed. Matrix…
We investigate the spectral distribution of random matrix ensembles with correlated entries. We consider symmetric matrices with real valued entries and stochastically independent diagonals. Along the diagonals the entries may be…
We study the performance of transformers as a function of the number of repetitions of training examples with algorithmically generated datasets. On three problems of mathematics: the greatest common divisor, modular multiplication, and…
We study continuity of the roots of nonmonic polynomials as a function of their coefficients using only the most elementary results from an introductory course in real analysis and the theory of single variable polynomials. Our approach…
This paper investigates a statistical procedure for testing the equality of two independent estimated covariance matrices when the number of potentially dependent data vectors is large and proportional to the size of the vectors, that is,…
A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts, or unmodeled temporal effects. We develop and…
We study heavy-tailed Hermitian random matrices that are unitarily invariant. The invariance implies that the eigenvalue and eigenvector statistics are decoupled. The motivating question has been whether a freely stable random matrix has…
In the last several years, the intimate connection between convex optimization and learning problems, in both statistical and sequential frameworks, has shifted the focus of algorithmic machine learning to examine this interplay. In…
Regression models usually tend to recover a noisy signal in the form of a combination of regressors, also called features in machine learning, themselves being the result of a learning process.The alignment of the prior covariance feature…
We study the typical properties of polynomial Support Vector Machines within a Statistical Mechanics approach that allows us to analyze the effect of different normalizations of the features. If the normalization is adecuately chosen, there…
Learning representations that capture the underlying data generating process is a key problem for data efficient and robust use of neural networks. One key property for robustness which the learned representation should capture and which…
We introduce the notion of a reproducible algorithm in the context of learning. A reproducible learning algorithm is resilient to variations in its samples -- with high probability, it returns the exact same output when run on two samples…
An important characteristic of neural networks is their ability to learn representations of the input data with effective features for prediction, which is believed to be a key factor to their superior empirical performance. To better…
Let $f=(f_1,\ldots,f_n)$ be a system of $n$ complex homogeneous polynomials in $n$ variables of degree $d$. We call $\lambda\in\mathbb{C}$ an eigenvalue of $f$ if there exists $v\in\mathbb{C}^n\backslash\{0\}$ with $f(v)=\lambda v$,…
Learning with limited labelled data, such as prompting, in-context learning, fine-tuning, meta-learning or few-shot learning, aims to effectively train a model using only a small amount of labelled samples. However, these approaches have…
We study the spectral properties of a class of random matrices where the matrix elements depend exponentially on the distance between uniformly and randomly distributed points. This model arises naturally in various physical contexts, such…