Related papers: Stochastic weight matrix dynamics during learning …
Transport phenomena are ubiquitous in nature and known to be important for various scientific domains. Examples can be found in physics, electrochemistry, heterogeneous catalysis, physiology, etc. To obtain new information about diffusive…
In this paper we study a parametric class of stochastic processes to model both fast and slow anomalous diffusion. This class, called generalized grey Brownian motion (ggBm), is made up off self-similar with stationary increments processes…
It was shown roughly thirty years ago that the density correlations of eigenvalues of large random matrices display a universal form, independent of most of the details of the distribution of the random matrix itself. We show that when the…
We take a new look at the problem of disentangling the volatility and jumps processes of daily stock returns. We first provide a computational framework for the univariate stochastic volatility model with Poisson-driven jumps that offers a…
State space models (SSMs) have shown remarkable empirical performance on many long sequence modeling tasks, but a theoretical understanding of these models is still lacking. In this work, we study the learning dynamics of linear SSMs to…
We investigate whether the Wigner semi-circle and Marcenko-Pastur distributions, often used for deep neural network theoretical analysis, match empirically observed spectral densities. We find that even allowing for outliers, the observed…
We consider the problem of learning a Gaussian variational approximation to the posterior distribution for a high-dimensional parameter, where we impose sparsity in the precision matrix to reflect appropriate conditional independence…
We propose new limiting dynamics for stochastic gradient descent in the small learning rate regime called stochastic modified flows. These SDEs are driven by a cylindrical Brownian motion and improve the so-called stochastic modified…
Background: Deep learning models are typically trained using stochastic gradient descent or one of its variants. These methods update the weights using their gradient, estimated from a small fraction of the training data. It has been…
We develop an approach to learn an interpretable semi-parametric model of a latent continuous-time stochastic dynamical system, assuming noisy high-dimensional outputs sampled at uneven times. The dynamics are described by a nonlinear…
Learning in Deep Neural Networks (DNN) takes place by minimizing a non-convex high-dimensional loss function, typically by a stochastic gradient descent (SGD) strategy. The learning process is observed to be able to find good minimizers…
Increasing effort is put into the development of methods for learning mechanistic models from data. This task entails not only the accurate estimation of parameters but also a suitable model structure. Recent work on the discovery of…
In this study, we develop a stochastic optimal control approach with reinforcement learning structure to learn the unknown parameters appeared in the drift and diffusion terms of the stochastic differential equation. By choosing an…
Generalizing manipulation skills to new situations requires extracting invariant patterns from demonstrations. For example, the robot needs to understand the demonstrations at a higher level while being invariant to the appearance of the…
Given the variable-speed random walk on a weighted graph and a metric adapted to the structure of the random walk, we construct a Brownian motion on a closely related metric graph which behaves similarly to the VSRW and for which the…
Score-based diffusion models currently constitute the state of the art in continuous generative modeling. These methods are typically formulated via overdamped or underdamped Ornstein--Uhlenbeck-type stochastic differential equations, in…
We develop an information-theoretic formulation of stochastic dynamics in which the fundamental stochastic variable is the total action connecting spacetime points, rather than individual paths. By maximizing Shannon entropy over a joint…
Deep learning-based multivariate and multistep-ahead traffic forecasting models are typically trained with the mean squared error (MSE) or mean absolute error (MAE) as the loss function in a sequence-to-sequence setting, simply assuming…
In this work, multiplicative stochasticity is applied to the learning rate of stochastic optimization algorithms, giving rise to stochastic learning-rate schemes. In-expectation theoretical convergence results of Stochastic Gradient Descent…
This paper is the second of a series devoted to the study of the dynamics of the spectrum of large random matrices. We study general extensions of the partial differential equation arising to characterize the limit spectral measure of the…