相关论文: Noether: The More Things Change, the More Stay the…

Noether's theorem relates constants of motion to the symmetries of the system. Here we investigate a manifestation of Noether's theorem in non-Hermitian systems, where the inner product is defined differently from quantum mechanics. In this…

The problem of detecting and quantifying the presence of symmetries in datasets is useful for model selection, generative modeling, and data analysis, amongst others. While existing methods for hard-coding transformations in neural networks…

In a physical neural system, learning rules must be local both in space and time. In order for learning to occur, non-local information must be communicated to the deep synapses through a communication channel, the deep learning channel. We…

We study two factors in neural network training: data parallelism and sparsity; here, data parallelism means processing training data in parallel using distributed systems (or equivalently increasing batch size), so that training can be…

Deep neural networks are workhorse models in machine learning with multiple layers of non-linear functions composed in series. Their loss function is highly non-convex, yet empirically even gradient descent minimisation is sufficient to…

Deep neural network training spends most of the computation on examples that are properly handled, and could be ignored. We propose to mitigate this phenomenon with a principled importance sampling scheme that focuses computation on…

Symmetries play a major role in physics, in particular since the work by E. Noether and H. Weyl in the first half of last century. Herein, we briefly review their role by recalling how symmetry changes allow to conceptually move from…

This paper advances the study of multivariate function approximation using neural network operators activated by symmetrized and perturbed hyperbolic tangent functions. We propose new non-linear operators that preserve dynamic symmetries…

Continual learning, the ability of a model to adapt to an ongoing sequence of tasks without forgetting earlier ones, is a central goal of artificial intelligence. To better understand its underlying mechanisms, we study the limitations of…

Emmy Noether proved two deep theorems, and their converses, on the connection between symmetries and conservation laws. Because these theorems are not in the mainstream of her scholarly work, which was the development of modern abstract…

Statistical neurodynamics studies macroscopic behaviors of randomly connected neural networks. We consider a deep layered feedforward network where input signals are processed layer by layer. The manifold of input signals is embedded in a…

We perform an experimental study of the dynamics of Stochastic Gradient Descent (SGD) in learning deep neural networks for several real and synthetic classification tasks. We show that in the initial epochs, almost all of the performance…

In this paper we study symmetries, Newtonoid vector fields, conservation laws, Noether's Theorem and its converse, in the framework of the $k$-symplectic formalism, using the Fr\"olicher-Nijenhuis formalism on the space of $k^1$-velocities…

Equivariant neural networks have proven to be effective for tasks with known underlying symmetries. However, optimizing equivariant networks can be tricky and best training practices are less established than for standard networks. In…

The relation between symmetries and local conservation laws, known as Noether's theorem, plays an important role in modern theoretical physics. As a discrete analog of the differentiable physical system, a good numerical scheme should admit…

Recently, Hamiltonian neural networks (HNN) have been introduced to incorporate prior physical knowledge when learning the dynamical equations of Hamiltonian systems. Hereby, the symplectic system structure is preserved despite the…

Neural networks are trained by optimizing multi-dimensional sets of fitting parameters on non-convex loss landscapes. Low-loss regions of the landscapes correspond to the parameter sets that perform well on the training data. A key issue in…

We consider Noether symmetries of the equations defined by the sections of characteristic line bundles of nondegenerate 1-forms and of the associated perturbed systems. It appears that this framework can be used for time-dependent systems…

A didactic and systematic derivation of Noether point symmetries and conserved currents is put forward in special relativistic field theories, without a priori assumptions about the transformation laws. Given the Lagrangian density, the…

Adaptive optimizers can reduce to normalized steepest descent (NSD) when only adapting to the current gradient, suggesting a close connection between the two algorithmic families. A key distinction between their analyses, however, lies in…