Related papers: Stochastic weight matrix dynamics during learning …

In this thesis, we extend the recently introduced theory of stochastic modified equations (SMEs) for stochastic gradient optimization algorithms. In Ch. 3 we study time-inhomogeneous SDEs driven by Brownian motion. For certain SDEs we prove…

We analyze the microscopic model of quantum Brownian motion, describing a Brownian particle interacting with a bosonic bath through a coupling which is linear in the creation and annihilation operators of the bath, but may be a nonlinear…

A dynamic Boltzmann machine (DyBM) has been proposed as a model of a spiking neural network, and its learning rule of maximizing the log-likelihood of given time-series has been shown to exhibit key properties of spike-timing dependent…

In this paper we prove the convergence to the stochastic Burgers equation from one-dimensional interacting particle systems, whose dynamics allow the degeneracy of the jump rates. To this aim, we provide a new proof of the second order…

We introduce a novel class of Bayesian mixtures for normal linear regression models which incorporates a further Gaussian random component for the distribution of the predictor variables. The proposed cluster-weighted model aims to…

We conjecture that the inherent difference in generalisation between adaptive and non-adaptive gradient methods in deep learning stems from the increased estimation noise in the flattest directions of the true loss surface. We demonstrate…

Stochastic mechanics is based on the hypothesis that all matter is subject to universal modified Brownian motion. In this report, we calculated probability density distributions using concepts of stochastic mechanics independent of…

In this paper we study the dynamics of stochastic microorganism flocculation models. Given the strong influence of environmental and seasonal fluctuations that are present in these models, we propose a stochastic model that includes…

Statistical (machine learning) tools for equation discovery require large amounts of data that are typically computer generated rather than experimentally observed. Multiscale modeling and stochastic simulations are two areas where learning…

Understanding the inductive bias and generalization properties of large overparametrized machine learning models requires to characterize the dynamics of the training algorithm. We study the learning dynamics of large two-layer neural…

Learning unknown stochastic differential equations (SDEs) from observed data is a significant and challenging task with applications in various fields. Current approaches often use neural networks to represent drift and diffusion functions,…

Given a quantum many-body system and the expectation-value dynamics of some operator, we study how this reference dynamics is altered due to a perturbation of the system's Hamiltonian. Based on projection operator techniques, we unveil that…

An important question in deep learning is how higher-order optimization methods affect generalization. In this work, we analyze a stochastic Gauss-Newton (SGN) method with Levenberg-Marquardt damping and mini-batch sampling for training…

This paper presents a solution for persistent monitoring of real-world stochastic phenomena, where the underlying covariance structure changes sharply across time, using a small number of mobile robot sensors. We propose an adaptive…

Grokking is the intriguing phenomenon where a model learns to generalize long after it has fit the training data. We show both analytically and numerically that grokking can surprisingly occur in linear networks performing linear tasks in a…

We illustrate the stochastic method for solving the Schwinger-Dyson equations in large-N quantum field theories described in ArXiv:1009.4033 on the example of the Gross-Witten unitary matrix model. In the strong-coupling limit, this method…

We study the learning dynamics of a multi-pass, mini-batch Stochastic Gradient Descent (SGD) procedure for empirical risk minimization in high-dimensional multi-index models with isotropic random data. In an asymptotic regime where the…

Stochastic transport due to a velocity field modeled by the superposition of small-scale divergence free vector fields activated by Fractional Gaussian Noises (FGN) is numerically investigated. We present two non-trivial contributions: the…

We develop a master equation formalism to describe the evolution of the average density matrix of a closed quantum system driven by a stochastic Hamiltonian. The average over random processes generally results in decoherence effects in…

The Boltzmann Machine (BM) is a neural network composed of stochastically firing neurons that can learn complex probability distributions by adapting the synaptic interactions between the neurons. BMs represent a very generic class of…