Related papers: Small Singular Values Matter: A Random Matrix Anal…
The releases of powerful open-weight large language models (LLMs) are often not accompanied by access to their full training data. Existing interpretability methods, particularly those based on activations, often require or assume…
Power systems are developing very fast nowadays, both in size and in complexity; this situation is a challenge for Early Event Detection (EED). This paper proposes a data- driven unsupervised learning method to handle this challenge.…
We obtain lower tail estimates for the smallest singular value of random matrices with independent but non-identically distributed entries. Specifically, we consider $n\times n$ matrices with complex entries of the form \[ M = A\circ X + B…
Random Matrix Theory (RMT) is capable of making predictions for the spectral fluctuations of a physical system only after removing the influence of the level density by unfolding the spectra. When the level density is known, unfolding is…
We study high-dimensional sample covariance matrices based on independent random vectors with missing coordinates. The presence of missing observations is common in modern applications such as climate studies or gene expression…
We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast…
Motivated by recent studies of the phenomenon of Coherent Perfect Absorption, we develop the random matrix theory framework for understanding statistics of the zeros of the (subunitary) scattering matrices in the complex energy plane, as…
We investigate the universality of singular value and eigenvalue distributions of matrix valued functions of independent random matrices and apply these general results in several examples. In particular we determine the limit distribution…
Consider the problem of estimating the entries of a large matrix, when the observed entries are noisy versions of a small random fraction of the original entries. This problem has received widespread attention in recent times, especially…
Memory constraint of always-on devices is one of the major concerns when deploying speech processing models on these devices. While larger models trained with sufficiently large amount of data generally perform better, making them fit in…
We study general singular value shrinkage estimators in high-dimensional regression and classification, when the number of features and the sample size both grow proportionally to infinity. We allow models with general covariance matrices…
Much research effort has been devoted to explaining the success of deep learning. Random Matrix Theory (RMT) provides an emerging way to this end: spectral analysis of large random matrices involved in a trained deep neural network (DNN)…
Random matrix theory is finding an increasing number of applications in the context of information theory and communication systems, especially in studying the properties of complex networks. Such properties include short-term and long-term…
The present work provides an original framework for random matrix analysis based on revisiting the concentration of measure theory from a probabilistic point of view. By providing various notions of vector concentration ($q$-exponential,…
It has been hypothesized that neural networks with similar architectures trained on similar data learn shared representations relevant to the learning task. We build on this idea by extending the conceptual framework where representations…
Invisible units mainly refer to small-scale units that are not monitored by, and thus are not visible to utilities. Integration of these invisible units into power systems does significantly affect the way in which a distribution grid is…
Transformer-based models have recently become wildly successful across a diverse set of domains. At the same time, recent work has shown empirically and theoretically that Transformers are inherently limited. Specifically, they argue that…
The large-scale focusing inversion of gravity and magnetic potential field data using $L_1$-norm regularization is considered. The use of the randomized singular value decomposition methodology facilitates tackling the computational…
Random tensor networks are a powerful toy model for understanding the entanglement structure of holographic quantum gravity. However, unlike holographic quantum gravity, their entanglement spectra are flat. It has therefore been argued that…
Recent work on permutation-based model merging has shown impressive low- or zero-barrier mode connectivity between models from completely different initializations. However, this line of work has not yet extended to the Transformer…