Related papers: The Underlying Scaling Laws and Universal Statisti…
We study the convergence properties of a pair of learning algorithms (learning with and without memory). This leads us to study the dominant eigenvalue of a class of random matrices. This turns out to be related to the roots of the…
Scaling laws in deep learning -- empirical power-law relationships linking model performance to resource growth -- have emerged as simple yet striking regularities across architectures, datasets, and tasks. These laws are particularly…
Size varies. Small things are typically more frequent than large things. The logarithm of frequency often declines linearly with the logarithm of size. That power law relation forms one of the common patterns of nature. Why does the…
Multilayer networks are widespread in natural and manmade systems. Key properties of these networks are their spectral and eigenfunction characteristics, as they determine the critical properties of many dynamics occurring on top of them.…
A new model ecosystem consisting of many interacting species is introduced. The species are connected through a random matrix with a given connectivity. It is shown that the system is organized close to a boundary of marginal stability in…
In order to pursue the issue of the relation between the financial cross-correlations and the conventional Random Matrix Theory we analyse several characteristics of the stock market correlation matrices like the distribution of…
Statistical laws describe regular patterns observed in diverse scientific domains, ranging from the magnitude of earthquakes (Gutenberg-Richter law) and metabolic rates in organisms (Kleiber's law), to the frequency distribution of words in…
A set of general allometric scaling laws is derived for different systems represented by tree networks. The formulation postulates self-similar networks with an arbitrary number of branches developed in each generation, and with an…
We consider a class of real numbers, a subset of irrational numbers and certain mathematical constants, for which the elements in the simple continued fraction appears to be random. As an illustrative example, one can consider $\pi = \{x_0,…
Using the diagrammatic method, we derive a set of self-consistent equations that describe eigenvalue distributions of large correlated asymmetric random matrices. The matrix elements can have different variances and be correlated with each…
The scaling law, a cornerstone of Large Language Model (LLM) development, predicts improvements in model performance with increasing computational resources. Yet, while empirically validated, its theoretical underpinnings remain poorly…
Random Matrix Theory (RMT) is applied to analyze the weight matrices of Deep Neural Networks (DNNs), including both production quality, pre-trained models such as AlexNet and Inception, and smaller models trained from scratch, such as…
A wide variety of complex networks (social, biological, information etc.) exhibit local clustering with substantial variation in the clustering coefficient (the probability of neighbors being connected). Existing models of large graphs…
Using numerical simulations and scaling theory we study the dynamics of the world-wide Web from the growth rules recently proposed in Ref. [1] with appropriate parameters. We demonstrate that the emergence of power-law behavior of the out-…
Map generalization is a process of producing maps at different levels of detail by retaining essential properties of the underlying geographic space. In this paper, we explore how the map generalization process can be guided by the…
Investigating the dynamics of learning in machine learning algorithms is of paramount importance for understanding how and why an approach may be successful. The tools of physics and statistics provide a robust setting for such…
We propose a generalization of the random matrix theory following the basic prescription of the recently suggested concept of superstatistics. Spectral characteristics of systems with mixed regular-chaotic dynamics are expressed as weighted…
Results on the spectral behavior of random matrices as the dimension increases are applied to the problem of detecting the number of sources impinging on an array of sensors. A common strategy to solve this problem is to estimate the…
Neural scaling laws characterize how model performance improves as the model size scales up. Inspired by empirical observations, we introduce a resource model of neural scaling. A task is usually composite hence can be decomposed into many…
We study the universality of spectral statistics of large random matrices. We consider $N\times N$ symmetric, hermitian or quaternion self-dual random matrices with independent, identically distributed entries (Wigner matrices) where the…