Related papers: A gradient flow formulation for the stochastic Ama…
We study the dynamics of gradient flow for training a multi-head softmax attention model for in-context learning of multi-task linear regression. We establish the global convergence of gradient flow under suitable choices of initialization.…
Gaseous flows show a diverse set of behaviors on different characteristic scales. Given the coarse-grained modeling in theories of fluids, considerable uncertainties may exist between the flow-field solutions and the real physics. To study…
We develop a general analytical and numerical framework for estimating intra- and extra-neurite water fractions and diffusion coefficients, as well as neurite orientational dispersion, in each imaging voxel. By employing a set of rotational…
In this paper, we continue the development of a generic adiabatic scheme for nonlinear evolutions. We consider an abstract gradient flow of some energy functional, together with a given manifold of static solutions arising from broken…
For smooth vector fields the classical method of characteristics provides a link between the ordinary differential equation and the corresponding continuity equation (or transport equation). We study an analog of this connection for merely…
Strongly interacting quantum systems described by non-stoquastic Hamiltonians exhibit rich low-temperature physics. Yet, their study poses a formidable challenge, even for state-of-the-art numerical techniques. Here, we investigate…
A fundamental problem in neuroscience is to characterize the dynamics of spiking from the neurons in a circuit that is involved in learning about a stimulus or a contingency. A key limitation of current methods to analyze neural spiking…
We are interested in existence of gradient flows for shape functionals especially for first Laplacian eigenvalues. We introduce different techniques to prove existence and use different formulations for gradient flows. We apply a…
In the papers (Shvidler, 1985 and 1993, and Shvidler and Karasaki, 1999, 2001, 2005, and 2008) we developed an approach for finding the exactly averaged equations of flow and transport in porous media. We studied for steady state flow with…
Normalizing flows are a class of generative models that enable exact likelihood evaluation. While these models have already found various applications in particle physics, normalizing flows are not flexible enough to model many of the…
We consider optimizing two-layer neural networks in the mean-field regime where the learning dynamics of network weights can be approximated by the evolution in the space of probability measures over the weight parameters associated with…
We propose nonparametric estimators for the second-order central moments of possibly anisotropic spherical random fields, within a functional data analysis context. We consider a measurement framework where each random field among an…
Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow networks, which tend to get stuck in…
We propose a hybrid variational framework that enhances Neural Quantum States (NQS) with a Normalising Flow-based sampler to improve the expressivity and trainability of quantum many-body wavefunctions. Our approach decouples the sampling…
A machine learning method to predict steady external fluid flows using elliptic input features is introduced. Using data from as few as one high-fidelity simulation, the proposed method produces models generalizable under changes to…
We develop a general theory of flows in the space of Riemannian metrics induced by neural network gradient descent. This is motivated in part by recent advances in approximating Calabi-Yau metrics with neural networks and is enabled by…
We prove the well-posedness of entropy solutions for a wide class of nonlocal transport equations with nonlinear mobility in one spatial dimension. The solution is obtained as the limit of approximations constructed via a deterministic…
The Neural Tangent Kernel (NTK) framework has provided deep insights into the training dynamics of neural networks under gradient flow. However, it relies on the assumption that the network is differentiable with respect to its parameters,…
This paper is concerned with a fourth order nonlinear dispersive partial differential equation for closed curve flow on a K\"ahler manifold. The main results is that the initial value problem has a solution locally in time if the K\"ahler…
This paper introduces an iterative algorithm for training nonparametric additive models that enjoys favorable memory storage and computational requirements. The algorithm can be viewed as the functional counterpart of stochastic gradient…