Related papers: The structure of conservative gradient fields
This paper considers non-smooth optimization problems where we seek to minimize the pointwise maximum of a continuously parameterized family of functions. Since the objective function is given as the solution to a maximization problem,…
Deep learning optimization exhibits structure that is not captured by worst-case gradient bounds. Empirically, gradients along training trajectories are often temporally predictable and evolve within a low-dimensional subspace. In this work…
A large class of first order partial nonlinear differential equations in two independent variables which possess an infinite set of polynomial conservation laws derived from an explicit generating function is constructed. The conserved…
Following ideas of Lawvere and Linton we prove that classical varieties are precisely the exact categories with a varietal generator. This means a strong generator which is abstractly finite and regularly projective. An analogous…
Conservation laws are considered to be fundamental laws of nature. It has broad applications in many fields, including physics, chemistry, biology, geology, and engineering. Solving the differential equations associated with conservation…
Any local field theory can be equivalently reformulated in the so-called unfolded form. General unfolded equations are non-Lagrangian even though the original theory is Lagrangian. Using the theory of a scalar field as a basic example, the…
Deep Learning's recent successes have mostly relied on Convolutional Networks, which exploit fundamental statistical properties of images, sounds and video data: the local stationarity and multi-scale compositional structure, that allows…
At present, object recognition studies are mostly conducted in a closed lab setting with classes in test phase typically in training phase. However, real-world problem is far more challenging because: i) new classes unseen in the training…
A general field theory for classical particle-field systems is developed. Compared with the standard classical field theory, the distinguish feature of a classical particle-field system is that the particles and fields reside on different…
This work attempts to interpret modern deep (convolutional) networks from the principles of rate reduction and (shift) invariant classification. We show that the basic iterative gradient ascent scheme for optimizing the rate reduction of…
We introduce a notion of generalized homogeneous derivations on graded rings as a natural extension of the homogeneous derivations defined by Kanunnikov. We then define gr-generalized derivations, which preserve the degrees of homogeneous…
A number of results have recently demonstrated the benefits of incorporating various constraints when training deep architectures in vision and machine learning. The advantages range from guarantees for statistical generalization to better…
One can see deep-learning models as compositions of functions within the so-called tame geometry. In this expository note, we give an overview of some topics at the interface of tame geometry (also known as o-minimality), optimization…
Contrastive learning effectively clusters data despite a loss landscape filled with poor solutions, a success that is heavily dependent on the choice of data augmentations. How optimization consistently finds meaningful patterns remains an…
In the paper it is shown that, even without a knowledge of the concrete form of the equations of mathematical physics and field theories, with the help of skew-symmetric differential forms one can see specific features of the equations of…
Electric fields are commonly visualized with field line diagrams, which only unambiguously specify the field's direction. We consider two simple questions. First, can one deduce if an electric field is conservative, as required e.g. in…
(Partial) differential equations (PDEs) are fundamental tools for describing natural phenomena, making their solution crucial in science and engineering. While traditional methods, such as the finite element method, provide reliable…
Data-centric methods have shown great potential in understanding and predicting spatiotemporal dynamics, enabling better design and control of the object system. However, deep learning models often lack interpretability, fail to obey…
Classic algorithms and machine learning systems like neural networks are both abundant in everyday life. While classic computer science algorithms are suitable for precise execution of exactly defined tasks such as finding the shortest path…
We propose and study a hierarchical algorithm to generate graphs having a predetermined distribution of cliques, the fully connected subgraphs. The construction mechanism may be either random or incorporate preferential attachment. We…