Related papers: Dynamical Behaviors of the Gradient Flows for In-C…
We present a novel deep learning framework for flow field predictions in irregular domains when the solution is a function of the geometry of either the domain or objects inside the domain. Grid vertices in a computational fluid dynamics…
We are interested in existence of gradient flows for shape functionals especially for first Laplacian eigenvalues. We introduce different techniques to prove existence and use different formulations for gradient flows. We apply a…
Accurate and efficient fluid flow models are essential for applications relating to many physical phenomena including geophysical, aerodynamic, and biological systems. While these flows may exhibit rich and multiscale dynamics, in many…
By means of a novel variational approach and using dual maps techniques and general ideas of dynamical system theory we derive exact results about several models of transport flows, for which we also obtain a complete description of their…
We introduce ImitationFlow, a novel Deep generative model that allows learning complex globally stable, stochastic, nonlinear dynamics. Our approach extends the Normalizing Flows framework to learn stable Stochastic Differential Equations.…
Sampling a target probability distribution with an unknown normalization constant is a fundamental challenge in computational science and engineering. Recent work shows that algorithms derived by considering gradient flows in the space of…
Interpreting gradient methods as fixed-point iterations, we provide a detailed analysis of those methods for minimizing convex objective functions. Due to their conceptual and algorithmic simplicity, gradient methods are widely used in…
We develop a general mathematical framework to analyze scaling regimes and derive explicit analytic solutions for gradient flow (GF) in large learning problems. Our key innovation is a formal power series expansion of the loss evolution,…
It is investigated a possibility of physical interpretation of vector fields (dynamic flows) in Euclidean spaces of higher dimension. There are analyzed the methods of measurements of dynamic flows, the characteristics of dynamic flow and…
The first part of this paper studies the evolution of gradient flow for homogeneous neural networks near a class of saddle points exhibiting a sparsity structure. The choice of these saddle points is motivated from previous works on…
Neural sequence models, especially transformers, exhibit a remarkable capacity for in-context learning. They can construct new predictors from sequences of labeled examples $(x, f(x))$ presented in the input without further parameter…
We propose a novel approach for learning the evolution that employs differentiable neural networks to approximate the full GENERIC structure. Instead of manually choosing the fitted parameters, we learn the whole model together with the…
We construct a kinetic model for matter-radiation interactions whose hydrodynamic gradient expansion can be computed analytically up to infinite order in derivatives, in the fully nonlinear regime, and for arbitrary flows. The frequency…
In deep learning, it is common to use more network parameters than training points. In such scenarioof over-parameterization, there are usually multiple networks that achieve zero training error so that thetraining algorithm induces an…
This paper studies interpretability of convolutional networks by means of saliency maps. Most approaches based on Class Activation Maps (CAM) combine information from fully connected layers and gradient through variants of backpropagation.…
Graph neural networks (GNNs) have shown promise in learning unstructured mesh-based simulations of physical systems, including fluid dynamics. In tandem, geometric deep learning principles have informed the development of equivariant…
Understanding the intricate non-convex training dynamics of softmax-based models is crucial for explaining the empirical success of transformers. In this article, we analyze the gradient flow dynamics of the value-softmax model, defined as…
In this book we collect many different and useful facts around gradient descent method. First of all we consider gradient descent with inexact oracle. We build a general model of optimized function that include composite optimization…
We analyze gradient flows with jumps generated by a finite set of complete vector fields in involution using some Radon measures $u\in \mathcal{U}_a$ as admissible perturbations. Both the evolution of a bounded gradient flow $\{x^u(t,\l)\in…
This paper considers continuously differentiable functions of two vector variables that have (possibly a continuum of) min-max saddle points. We study the asymptotic convergence properties of the associated saddle-point dynamics…