Related papers: String Method for Generalized Gradient Flows: Comp…
Understanding the geometry of learned distributions is fundamental to improving and interpreting diffusion models, yet systematic tools for exploring their landscape remain limited. Standard latent-space interpolations fail to respect the…
We study the implicit regularization of mini-batch stochastic gradient descent, when applied to the fundamental problem of least squares regression. We leverage a continuous-time stochastic differential equation having the same moments as…
Multiplicative cascades have been introduced in turbulence to generate random or deterministic fields having intermittent values and long-range power-law correlations. Generally this is done using discrete construction rules leading to…
Sampling a target probability distribution with an unknown normalization constant is a fundamental challenge in computational science and engineering. Recent work shows that algorithms derived by considering gradient flows in the space of…
Stochastic gradient methods are scalable for solving large-scale optimization problems that involve empirical expectations of loss functions. Existing results mainly apply to optimization problems where the objectives are one- or two-level…
This thesis develops exact analytical tools to study strongly correlated stochastic systems, with a focus on extreme value statistics, gap statistics, and full counting statistics in multi-particle processes. A central contribution is the…
Turbulent cascades characterize the transfer of energy injected by a random force at large scales towards the small scales. In hydrodynamic turbulence, when the Reynolds number is large, the velocity field of the fluid becomes irregular and…
We consider the stochastic gradient method with random reshuffling ($\mathsf{RR}$) for tackling smooth nonconvex optimization problems. $\mathsf{RR}$ finds broad applications in practice, notably in training neural networks. In this work,…
Random Reshuffling (RR) is an algorithm for minimizing finite-sum functions that utilizes iterative gradient descent steps in conjunction with data reshuffling. Often contrasted with its sibling Stochastic Gradient Descent (SGD), RR is…
Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering. This task may be cast as an optimization problem over all probability measures, and an initial…
In this paper, we study a simple hydrodynamical model showing abrupt flow reversals at random times. For a suitable range of parameters, we show that the dynamics of flow reversal is accurately described by stochastic differential…
Modern machine learning systems represent their computations as dataflow graphs. The increasingly complex neural network architectures crave for more powerful yet efficient programming abstractions. In this paper we propose an efficient…
Transition path sampling is a method for estimating the rates of rare events in molecular systems based on the gradual transformation of a path distribution containing a small fraction of reactive trajectories into a biased distribution in…
This paper is concerned with classes of models of stochastic reaction dynamics with time-scales separation. We demonstrate that the existence of the time-scale separation naturally leads to the application of the averaging principle and…
Random geometric graphs (RGG) can be formalized as hidden-variables models where the hidden variables are the coordinates of the nodes. Here we develop a general approach to extract the typical configurations of a generic hidden-variables…
The Cartesian reverse derivative is a categorical generalization of reverse-mode automatic differentiation. We use this operator to generalize several optimization algorithms, including a straightforward generalization of gradient descent…
We review two recently developed efficient methods for calculating rate constants of processes dominated by rare events in high-dimensional complex systems. The first is transition interface sampling (TIS), based on the measurement of…
We present a stochastic method for reconstructing missing spatial and velocity data along the trajectories of small objects passively advected by turbulent flows with a wide range of temporal or spatial scales, such as small balloons in the…
Gradient-based iterative optimization methods are the workhorse of modern machine learning. They crucially rely on careful tuning of parameters like learning rate and momentum. However, one typically sets them using heuristic approaches…
We present well-balanced, high-order, semi-discrete numerical schemes for one-dimensional blood flow models with discontinuous mechanical properties and algebraic source terms representing friction and gravity. While discontinuities in…