Related papers: The structure of conservative gradient fields

We introduce and study a new class of differential fields in positive characteristic. We call them separably differentially closed fields and demonstrate that they are the differential analogue of separably closed fields. We prove several…

We formulate semi-classical field theory as an approximate decoherence-free-subspace of a finite-dimensional quantum-gravity hilbert space. A complementarity construction can be realized as a unitary transformation which changes the…

We explore conditions for when the gradient of a deep declarative node can be approximated by ignoring constraint terms and still result in a descent direction for the global loss function. This has important practical application when…

This paper proposes a fractional order gradient method for the backward propagation of convolutional neural networks. To overcome the problem that fractional order gradient method cannot converge to real extreme point, a simplified…

Learning rules -- prescriptions for updating model parameters to improve performance -- are typically assumed rather than derived. Why do some learning rules work better than others, and under what assumptions can a given rule be considered…

Integrable difference equations commonly have more low-order conservation laws than occur for nonintegrable difference equations of similar complexity. We use this empirical observation to sift a large class of difference equations, in…

In the framework of the Stueckelberg-Wheeler-Feynman concept of a ``one-electron Universe'' we consider a worldline implicitly defined by a system of algebraic (precisely, polynomial) equations. Collection of pointlike ``particles'' of two…

The theories of strings and $D$-branes have motivated the development of non Abelian cohomology techniques in differential geometry, on the purpose to find a geometric interpretation of characteristic classes. The spaces studied here, like…

We introduce the notion of differential largeness for fields equipped with several commuting derivations (as an analogue to largeness of fields). We lay out the foundations of this new class of "tame" differential fields. We state several…

Examples exist of extended-real-valued closed functions on ${\bf R}^n$ whose subdifferentials (in the standard, limiting sense) have large graphs. By contrast, if such a function is semi-algebraic, then its subdifferential graph must have…

Backpropagation and the chain rule of derivatives have been prominent; however, the total derivative rule has not enjoyed the same amount of attention. In this work we show how the total derivative rule leads to an intuitive visual…

We propose a novel perspective to understand deep neural networks in an interpretable disentanglement form. For each semantic class, we extract a class-specific functional subnetwork from the original full model, with compressed structure…

We analyze the constant step size subgradient method on nonsmooth, nonconvex functions. We identify geometric assumptions on the objective function under which i) its domain admits a partition (stratification) into smooth manifolds (strata)…

We study algebras encoding stable range branching rules for the pairs of complex classical groups of the same type in the context of toric degenerations of spherical varieties. By lifting affine semigroup algebras constructed from…

Uncertainty estimation in large deep-learning models is a computationally challenging task, where it is difficult to form even a Gaussian approximation to the posterior distribution. In such situations, existing methods usually resort to a…

Although conservative Hamiltonian systems with constraints can be formulated in terms of Dirac structures, a more general framework is necessary to cover also dissipative systems such as gradient and metriplectic systems with constraints.…

We conservatively extend classical elementary differential calculus to the Cartesian closed category of convergence spaces. By specializing results about the convergence space representation of directed graphs, we use Cayley graphs to…

Semiclassical approximation based on extracting a c-number classical component from quantum field is widely used in the quantum field theory. Semiclassical states are considered then as Gaussian wave packets in the functional Schrodinger…

In this paper, we consider a deterministic dynamic programming model, and derive the envelope theorem using the Clarke differential. Compared with previous research, we do not require differentiability, convexity, or boundedness.

The paradigm of differentiable programming has significantly enhanced the scope of machine learning via the judicious use of gradient-based optimization. However, standard differentiable programming methods (such as autodiff) typically…