Related papers: Stochastic weight matrix dynamics during learning …
Investigating the dynamics of learning in machine learning algorithms is of paramount importance for understanding how and why an approach may be successful. The tools of physics and statistics provide a robust setting for such…
During training, weight matrices in machine learning architectures are updated using stochastic gradient descent or variations thereof. In this contribution we employ concepts of random matrix theory to analyse the resulting stochastic…
We consider a toy model for the study of monitored dynamics in a many-body quantum systems. We study the stochastic Schrodinger equation resulting from the continuous monitoring with a rate $\Gamma$ of a random hermitian operator chosen at…
We define a message-passing algorithm for computing magnetizations in Restricted Boltzmann machines, which are Ising models on bipartite graphs introduced as neural network models for probability distributions over spin configurations. To…
This paper is the third chapter of three of the author's undergraduate thesis. In this paper, we study the convergence of local bulk statistics for linearized covariance matrices under Dyson's Brownian motion. We consider deterministic…
We offer an alternative viewpoint on Dyson's original paper regarding the application of Brownian motion to random matrix theory (RMT). In particular we show how one may use the same approach in order to study the stochastic motion in the…
Finding reduced models of spatially-distributed chemical reaction networks requires an estimation of which effective dynamics are relevant. We propose a machine learning approach to this coarse graining problem, where a maximum entropy…
Decentralized learning often involves a weighted global loss with heterogeneous node weights $\lambda$. We revisit two natural strategies for incorporating these weights: (i) embedding them into the local losses to retain a uniform weight…
We identify an issue in recent approaches to learning-based control that reformulate systems with uncertain dynamics using a stochastic differential equation. Specifically, we discuss the approximation that replaces a model with fixed but…
We show that the Dyson Brownian Motion exhibits local universality after a very short time assuming that local rigidity and level repulsion hold. These conditions are verified, hence bulk spectral universality is proven, for a large class…
A density matrix describes the statistical state of a quantum system. It is a powerful formalism to represent both the quantum and classical uncertainty of quantum systems and to express different statistical operations such as measurement,…
Quantum gravity has long remained elusive from an observational standpoint. Developing effective cosmological models motivated by the fundamental aspects of quantum gravity is crucial for bridging theory with observations. One key aspect is…
The majority of machine learning methods can be regarded as the minimization of an unavailable risk function. To optimize the latter, given samples provided in a streaming fashion, we define a general stochastic Newton algorithm and its…
An innovative extension of Geometric Brownian Motion model is developed by incorporating a weighting factor and a stochastic function modelled as a mixture of power and trigonometric functions. Simulations based on this Modified Brownian…
We present a theoretical analysis of some popular adaptive Stochastic Gradient Descent (SGD) methods in the small learning rate regime. Using the stochastic modified equations framework introduced by Li et al., we derive effective…
In this letter we present an analytic method for calculating the transition probability between two random Gaussian matrices with given eigenvalue spectra in the context of Dyson Brownian motion. We show that in the Coulomb gas language, in…
The choice of batch-size in a stochastic optimization algorithm plays a substantial role for both optimization and generalization. Increasing the batch-size used typically improves optimization but degrades generalization. To address the…
Distributions of data or sensory stimuli often enjoy underlying invariances. How and to what extent those symmetries are captured by unsupervised learning methods is a relevant question in machine learning and in computational neuroscience.…
Circular Dyson Brownian motion describes the Brownian dynamics of particles on a circle (periodic boundary conditions), interacting through a logarithmic, long-range two-body potential. Within the log-gas picture of random matrix theory, it…
We show that the global fluctuations of spectra of GOE and GUE matrices and their principal submatrices executing Dyson's Brownian motion are Gaussian in the limit of large matrix dimensions. For nested submatrices one obtains a limiting…