Related papers: The Hamilton-Jacobi Theory of Deep Learning

Neural networks that synergistically integrate data and physical laws offer great promise in modeling dynamical systems. However, iterative gradient-based optimization of network parameters is often computationally expensive and suffers…

Recent experiments have shown that training trajectories of multiple deep neural networks with different architectures, optimization algorithms, hyper-parameter settings, and regularization methods evolve on a remarkably low-dimensional…

It is well known that time dependent Hamilton-Jacobi-Isaacs partial differential equations (HJ PDE), play an important role in analyzing continuous dynamic games and control theory problems. An important tool for such problems when they…

We generalize the Hamiltonian Monte Carlo algorithm with a stack of neural network layers and evaluate its ability to sample from different topologies in a two dimensional lattice gauge theory. We demonstrate that our model is able to…

We study the problem of Hamiltonian structure learning from real-time evolution: given the ability to apply $e^{-\mathrm{i} Ht}$ for an unknown local Hamiltonian $H = \sum_{a = 1}^m \lambda_a E_a$ on $n$ qubits, the goal is to recover $H$.…

We study quantitative estimates of compactness in $\mathbf{W}^{1,1}_{loc}$ for the map $S_t$, $t>0$ that associates to every given initial data $u_0\in \mathrm{Lip}(\mathbb{R}^N)$ the corresponding solution $S_t u_0$ of a Hamilton-Jacobi…

We develop a discrete analogue of Hamilton-Jacobi theory in the framework of discrete Hamiltonian mechanics. The resulting discrete Hamilton-Jacobi equation is discrete only in time. We describe a discrete analogue of Jacobi's solution and…

In this paper, we establish the higher order convergence rates in periodic homogenization of viscous Hamilton-Jacobi equations, which is convex and grows quadratically in the gradient variable. We observe that although the nonlinear…

We introduce a new machinery to study the large time behavior for general classes of Hamilton--Jacobi type equations, which include degenerate parabolic equations and weakly coupled systems. We establish the convergence results by using the…

We study continuous dependence estimates for viscous Hamilton- Jacobi equations defined on a network Gamma. Given two Hamilton-Jacobi equations, we prove an estimate of the C2-norm of the difference between the corresponding solutions in…

We consider an optimal control on networks in the spirit of the works of Achdou et al. (2013) and Imbert et al. (2013). The main new feature is that there are entry (or exit) costs at the edges of the network leading to a possible…

Recent work has shown that biologically plausible Hebbian learning can be integrated with backpropagation learning (backprop), when training deep convolutional neural networks. In particular, it has been shown that Hebbian learning can be…

We present a framework for efficient extraction of the viscosity solutions of nonlinear Hamilton-Jacobi equations with convex Hamiltonians. These viscosity solutions play a central role in areas such as front propagation, mean-field games,…

The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in a neural ordinary differential equation (NODE) is considered, that means finding the weights of a residual network with time continuous…

Many optimal control problems are formulated as two point boundary value problems (TPBVPs) with conditions of optimality derived from the Hamilton-Jacobi-Bellman (HJB) equations. In most cases, it is challenging to solve HJBs due to the…

This paper investigates the learning theory of Transformer networks for regression tasks on the compact Euclidean domain $[0,1]^d$ and $d$-dimensional compact Riemannian manifolds. We propose a novel constructive approximation framework for…

Reduction theory has played a major role in the study of Hamiltonian systems. On the other hand, the Hamilton-Jacobi theory is one of the main tools to integrate the dynamics of certain Hamiltonian problems and a topic of research on its…

Understanding natural symmetries is key to making sense of our complex and ever-changing world. Recent work has shown that neural networks can learn such symmetries directly from data using Hamiltonian Neural Networks (HNNs). But HNNs…

In this paper we present a general framework that allows one to study discretization of certain dynamical systems. This generalizes earlier work on discretization of Lagrangian and Hamiltonian systems on tangent bundles and cotangent…

Deep learning involves a difficult non-convex optimization problem with a large number of weights between any two adjacent layers of a deep structure. To handle large data sets or complicated networks, distributed training is needed, but…