Related papers: Dynamical Behaviors of the Gradient Flows for In-C…
A nonlinear parabolic equation of sixth order is analyzed. The equation arises as a reduction of a model from quantum statistical mechanics, and also as the gradient flow of a second-order information functional with respect to the…
This paper proposes a novel perspective on learning, positing it as the pursuit of dynamical invariants -- data combinations that remain constant or exhibit minimal change over time as a system evolves. This concept is underpinned by both…
The process of training an artificial neural network involves iteratively adapting its parameters so as to minimize the error of the network's prediction, when confronted with a learning task. This iterative change can be naturally…
Flows on surfaces are one of the most fundamental and classical objects in dynamical systems, and are studied from various areas (e.g. integrable systems, differential equations, fluid mechanics). Though hyperbolic flows and recurrent flows…
These are lecture notes for various Summer and Winter schools that I have given. The notes describe the methodology called Variational Modelling, and focus on the application to the modelling of gradient-flow systems. I describe the…
We use differential equations based approaches to provide some {\it \textbf{physics}} insights into analyzing the dynamics of popular optimization algorithms in machine learning. In particular, we study gradient descent, proximal gradient…
We study the convergence of gradient flows related to learning deep linear neural networks (where the activation function is the identity map) from data. In this case, the composition of the network layers amounts to simply multiplying the…
The scarcity of labeled data is a long-standing challenge for many machine learning tasks. We propose our gradient flow method to leverage the existing dataset (i.e., source) to generate new samples that are close to the dataset of interest…
This paper considers the problem of regulating a linear dynamical system to the solution of a convex optimization problem with an unknown or partially-known cost. We design a data-driven feedback controller - based on gradient flow dynamics…
Directly parameterizing and learning gradients of functions has widespread significance, with specific applications in inverse problems, generative modeling, and optimal transport. This paper introduces gradient networks (GradNets): novel…
Various results for higher-order perturbative calculations in the gradient-flow formalism are reviewed, including the gradient-flow beta function and the small-flow-time expansion of the hadronic vacuum polarization and the energy-momentum…
Traditional theories of optimization cannot describe the dynamics of optimization in deep learning, even in the simple setting of deterministic training. The challenge is that optimizers typically operate in a complex, oscillatory regime…
The dynamics of gradient and Hamiltonian flows with particular application to flows on adjoint orbits of a Lie group and the extension of this setting to flows on a loop group are discussed. Different types of gradient flows that arise from…
We consider generalized gradients in the general context of $G$-structures. They are natural first order differential operators acting on sections of vector bundles associated to irreducible $G$-representations. We study their geometric…
A machine learning method to predict steady external fluid flows using elliptic input features is introduced. Using data from as few as one high-fidelity simulation, the proposed method produces models generalizable under changes to…
Studying various functionals and associated gradient ows are known problems in differential geometry. The perpose of this article is to provide a general overview of curvature functionals in Finsler geometry and use their information for…
In this paper, the inherent gradient flow structures of thermo-poro-visco-elastic processes in porous media are examined for the first time. In the first part, a modelling framework is introduced aiming for describing such processes as…
Graph Neural Networks (GNNs) are powerful tools for addressing learning problems on graph structures, with a wide range of applications in molecular biology and social networks. However, the theoretical foundations underlying their…
The ability of learning useful features is one of the major advantages of neural networks. Although recent works show that neural network can operate in a neural tangent kernel (NTK) regime that does not allow feature learning, many works…
The capability of predicting environmental dynamics underpins both biological neural systems and general embodied AI in adapting to their surroundings. Yet prevailing approaches rest on static world models that falter when confronted with…