相关论文: Geometric Regularization from Overparameterization

We construct a new version of the higher covariant derivative regularization for a general ${\cal N}=2$ supersymmetric gauge theory formulated in terms of ${\cal N}=1$ superfields. This regularization preserves both supersymmetries of the…

The ability of overparameterized deep networks to interpolate noisy data, while at the same time showing good generalization performance, has been recently characterized in terms of the double descent curve for the test error. Common…

A fairly comprehensive analysis is presented for the gradient descent dynamics for training two-layer neural network models in the situation when the parameters in both layers are updated. General initialization schemes as well as general…

The impressive success of modern deep neural networks on computer vision tasks has been achieved through models of very large capacity compared to the number of available training examples. This overparameterization is often said to be…

Inspired by the remarkable success of large neural networks, there has been significant interest in understanding the generalization performance of over-parameterized models. Substantial efforts have been invested in characterizing how…

Recent works have highlighted scale invariance or symmetry present in the weight space of a typical deep network and the adverse effect it has on the Euclidean gradient based stochastic gradient descent optimization. In this work, we show…

In order to tackle the difficulty associated with the ill-posed nature of the image registration problem, regularization is often used to constrain the solution space. For most learning-based registration approaches, the regularization…

This theoretical paper is devoted to developing a rigorous theory for demystifying the global convergence phenomenon in a challenging scenario: learning over-parameterized Rectified Linear Unit (ReLU) nets for very high dimensional dataset…

Overparametrized neural networks trained by gradient descent (GD) can provably overfit any training data. However, the generalization guarantee may not hold for noisy data. From a nonparametric perspective, this paper studies how well…

A natural geometry, arising from the embedding into a Hilbert space of the parametrised probability measure for a given lattice model, is used to study the symmetry properties of real-space renormalisation group (RG) flow. In the projective…

In this paper, we consider linear ill-posed problems in Hilbert spaces and their regularization via frame decompositions, which are generalizations of the singular-value decomposition. In particular, we prove convergence for a general class…

The recently established metric reduction in generalized geometry is encoded in 0-dimensional supersymmetric $\sigma$-models. This is an example of balanced topological field theories. To find the geometric content of such models, the…

Matrix models of 2D quantum gravity are either exactly solvable for matter of central charge $ c\leq 1, $ or not understood. It would be useful to devise an approximate scheme which would be reasonable for the known cases and could be…

It is well known that popular optimization techniques can lead to overfitting or even a lack of convergence altogether; thus, practitioners often utilize ad hoc regularization terms added to the energy functional. When carefully crafted,…

A quadratic approximation of neural network loss landscapes has been extensively used to study the optimization process of these networks. Though, it usually holds in a very small neighborhood of the minimum, it cannot explain many…

Neural scaling laws and double-descent phenomena suggest that deep-network training obeys a simple macroscopic structure despite highly nonlinear optimization dynamics. We derive such structure directly from gradient descent in function…

Modern machine learning models are often trained in a setting where the number of parameters exceeds the number of training samples. To understand the implicit bias of gradient descent in such overparameterized models, prior work has…

The strong correlation between neurons or filters can significantly weaken the generalization ability of neural networks. Inspired by the well-known Tammes problem, we propose a novel diversity regularization method to address this issue,…

Variational regularization of ill-posed inverse problems is based on minimizing the sum of a data fidelity term and a regularization term. The balance between them is tuned using a positive regularization parameter, whose automatic choice…

In one dimension, any disorder is traditionally believed to localize all states. We show that this paradigm breaks down under hyperuniform disorder, which suppresses long-wavelength fluctuations and interpolates between random and periodic…