Related papers: Stochastic weight matrix dynamics during learning …
Deep kernel learning combines the non-parametric flexibility of kernel methods with the inductive biases of deep learning architectures. We propose a novel deep kernel learning model and stochastic variational inference procedure which…
This paper studies continuous-time stochastic control problems whose controlled states are fully non-Markovian and depend on unknown model parameters. Such problems arise naturally in path-dependent stochastic differential equations,…
As the size of modern data sets exceeds the disk and memory capacities of a single computer, machine learning practitioners have resorted to parallel and distributed computing. Given that optimization is one of the pillars of machine…
We provide an algorithm based on weighted-ensemble (WE) methods, to accurately sample systems at steady state. Applying our method to different one- and two-dimensional models, we succeed to calculate steady state probabilities of order…
Given a segment of time series of a system at a particular set of parameter values, can one infers the global behavior of the system in its parameter space? Here we show that by using a learning machine we can achieve such a goal to a…
The Wigner-Gaudin-Mehta-Dyson conjecture asserts that the local eigenvalue statistics of large random matrices exhibit universal behavior depending only on the symmetry class of the matrix ensemble. For invariant matrix models, the…
We look at the equilibrium of a Brownian particle in an inhomogeneous space following the alternative approach proposed in ref.[1]. We consider a coordinate dependent damping that makes the stochastic dynamics the one with multiplicative…
Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for…
Graphical models are powerful tools for modeling high-dimensional data, but learning graphical models in the presence of latent variables is well-known to be difficult. In this work we give new results for learning Restricted Boltzmann…
Stochastic gradient descent plays a fundamental role in nearly all applications of deep learning. However its ability to converge to a global minimum remains shrouded in mystery. In this paper we propose to study the behavior of the loss…
We discuss the geometric foundation behind the use of stochastic processes in the frame bundle of a smooth manifold to build stochastic models with applications in statistical analysis of non-linear data. The transition densities for the…
We study the problem of system identification for stochastic continuous-time dynamics, based on a single finite-length state trajectory. We present a method for estimating the possibly unstable open-loop matrix by employing properly…
We develop and analyze a broad family of stochastic/randomized algorithms for inverting a matrix. We also develop specialized variants maintaining symmetry or positive definiteness of the iterates. All methods in the family converge…
We present a learning theory for the training of a linear system operator having an input compositional variable and propose a Bayesian inversion method for inferring the unknown variable from an output of a noisy linear system. We assume…
We prove the Wigner-Dyson-Mehta conjecture at fixed energy in the bulk of the spectrum for generalized symmetric and Hermitian Wigner matrices. Previous results concerning the universality of random matrices either require an averaging in…
The circular Dyson Brownian motion model refers to the stochastic dynamics of the log-gas on a circle. It also specifies the eigenvalues of certain parameter-dependent ensembles of unitary random matrices. This model is considered with the…
The asymptotic behavior of the stochastic gradient algorithm with a biased gradient estimator is analyzed. Relying on arguments based on the dynamic system theory (chain-recurrence) and the differential geometry (Yomdin theorem and…
One century after Einstein's work, Brownian Motion still remains both a fundamental open issue and a continous source of inspiration for many areas of natural sciences. We first present a discussion about stochastic and deterministic…
We consider a general Langevin dynamics for the one-dimensional N-particle Coulomb gas with confining potential $V$ at temperature $\beta$. These dynamics describe for $\beta=2$ the time evolution of the eigenvalues of $N\times N$ random…
Recent work has established an empirically successful framework for adapting learning rates for stochastic gradient descent (SGD). This effectively removes all needs for tuning, while automatically reducing learning rates over time on…