Related papers: Mathematics of learning
A cornerstone of human statistical learning is the ability to extract temporal regularities / patterns from random sequences. Here we present a method of computing pattern time statistics with generating functions for first-order Markov…
Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in…
In this work, we investigate the convergence of numerical approximations to coercivity constants of variational problems. These constants are essential components of rigorous error bounds for reduced-order modeling; extension of these…
We give an upper bound on the total variation distance between the linear eigenvalue statistic, properly scaled and centred, of a random matrix with a variance profile and the standard Gaussian random variable. The second order Poincar\'e…
We study the eigenvalue problem for some special class of anti-triangular matrices. Though the eigenvalue problem is quite classical, as far as we know, almost nothing is known about properties of eigenvalues for anti-triangular matrices.…
We examine the empirical distribution of the eigenvalues and the eigenvectors of adjacency matrices of sparse regular random graphs. We find that when the degree sequence of the graph slowly increases to infinity with the number of…
In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning. We show that averaging gradients across examples -- akin to a logical OR of patterns -- can favor memorization and…
Methods for the reduction of the complexity of computational problems are presented, as well as their connections to renormalization, scaling, and irreversible statistical mechanics. Several statistically stationary cases are analyzed; for…
Random matrices arise in many mathematical contexts, and it is natural to ask about the properties that such matrices satisfy. If we choose a matrix with integer entries at random, for example, what is the probability that it will have a…
Machine learning is the study of computer algorithms that can automatically improve based on data and experience. Machine learning algorithms build a model from sample data, called training data, to make predictions or judgments without…
We consider a discrete, non-Hermitian random matrix model, which can be expressed as a shift of a rank-one perturbation of an anti-symmetric matrix. We show that, asymptotically almost surely, the real parts of the eigenvalues of the…
The paper addresses the calculation of correlation functions of permanental polynomials of matrices with random entries. By exploiting a convenient contour integral representation of the matrix permanent some explicit results are provided…
Generalization analyses of deep learning typically assume that the training converges to a fixed point. But, recent results indicate that in practice, the weights of deep neural networks optimized with stochastic gradient descent often…
The learning rate is one of the most important hyperparameters in deep learning, and how to control it is an active area within both AutoML and deep learning research. Approaches for learning rate control span from classic optimization to…
Conditional on the extended Riemann hypothesis, we show that with high probability, the characteristic polynomial of a random symmetric $\{\pm 1\}$-matrix is irreducible. This addresses a question raised by Eberhard in recent work. The main…
We demonstrate that a wide array of machine learning algorithms are specific instances of one single paradigm: reciprocal learning. These instances range from active learning over multi-armed bandits to self-training. We show that all these…
Acquisition of data is a difficult task in many applications of machine learning, and it is only natural that one hopes and expects the population risk to decrease (better performance) monotonically with increasing data points. It turns…
We investigate algebraic and arithmetic properties of a class of sequences of sparse polynomials that have binomial coefficients both as exponents and as coefficients. In addition to divisibility and irreducibility results we also consider…
The topic of this paper is the typical behavior of the spectral measures of large random matrices drawn from several ensembles of interest, including in particular matrices drawn from Haar measure on the classical Lie groups, random…
Synchronized measurements of a large power grid enable an unprecedented opportunity to study the spatialtemporal correlations. Statistical analytics for those massive datasets start with high-dimensional data matrices. Uncertainty is…