Related papers: The structure of conservative gradient fields

In this paper, we propose new structured second-order methods and structured adaptive-gradient methods obtained by performing natural-gradient descent on structured parameter spaces. Natural-gradient descent is an attractive approach to…

Evolutionary forms, as well as exterior forms, are skew-symmetric differential forms. But in contrast to the exterior forms, the basis of evolutionary forms is deforming manifolds (with unclosed metric forms). Such forms possess a…

Attention networks have proven to be an effective approach for embedding categorical inference within a deep neural network. However, for many tasks we may want to model richer structural dependencies without abandoning end-to-end training.…

Dickson's commutative semifields are an important class of finite division algebras. We generalise Dickson's construction of commutative division algebras by doubling both finite field extensions and central simple algebras and not…

The Cartesian reverse derivative is a categorical generalization of reverse-mode automatic differentiation. We use this operator to generalize several optimization algorithms, including a straightforward generalization of gradient descent…

We consider structure-preserving methods for conservative systems, which rigorously replicate the conservation property yielding better numerical solutions. There, corresponding to the skew-symmetry of the differential operator, that of…

Manifold learning approaches seek the intrinsic, low-dimensional data structure within a high-dimensional space. Mainstream manifold learning algorithms, such as Isomap, UMAP, $t$-SNE, Diffusion Map, and Laplacian Eigenmaps do not use data…

Given an objective function that is invariant under an action of a Lie group, we study how its subgradients relate to the orbits of the action. Our main finding is that they satisfy projection formulae analogous to those stemming from the…

The basis for the field theory are properties of the closed exterior differential forms (skew-symmetric differential forms defined on manifolds with the closed metric forms), which reflect properties of the conservation laws for physical…

We investigate the parameterization of deep neural networks that by design satisfy the continuity equation, a fundamental conservation law. This is enabled by the observation that any solution of the continuity equation can be represented…

A differential version of the classical Weil descent is established in all characteristics. It yields a theory of differential restriction of scalars for differential varieties over finite differential field extensions. This theory is then…

We develop a semi-discrete version of discrete variational mechanics with applications to numerical integration of classical field theories. The geometric preservation properties are studied.

We consider general, non-linear curvature perturbations on scales greater than the Hubble horizon scale by invoking an expansion in spatial gradients, the so-called gradient expansion. After reviewing the basic properties of the gradient…

We construct a natural branch divisor for equidimensional projective morphisms where the domain has lci singularities and the target is nonsingular. The method involves generalizing a divisor contruction of Mumford from sheaves to…

Simplicial complexes prove effective in modeling data with multiway dependencies, such as data defined along the edges of networks or within other higher-order structures. Their spectrum can be decomposed into three interpretable subspaces…

A complete proof of the recursive gradient approach is presented. It gives a construction of all the hierarchy structures of N=1 Super KdV, including the non-local one. A precise definition of the ring of superfields involved in the…

It is a fairly known fact that most of the algebras appearing in the theory of rings of differential operators, quantized algebras of different kinds (including many quantum groups), regular algebras in projective non-commutative geometry,…

We propose a categorical semantics of gradient-based machine learning algorithms in terms of lenses, parametrised maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it…

We introduce a method for training exactly conservative physics-informed neural networks and physics-informed deep operator networks for dynamical systems. The method employs a projection-based technique that maps a candidate solution…

We prove that if $A$ is a regular graded skew Clifford algebra and is a twist of a regular graded Clifford algebra $B$ by an automorphism, then the subalgebra of $A$ generated by a certain normalizing sequence of homogeneous degree-two…