Related papers: A New Angle on L2 Regularization
Due to the over-parameterization nature, neural networks are a powerful tool for nonlinear function approximation. In order to achieve good generalization on unseen data, a suitable inductive bias is of great importance for neural networks.…
We derive a divergence formula for a group of regularization methods with an L2 constraint. The formula is useful for regularization parameter selection, because it provides an unbiased estimate for the number of degrees of freedom. We…
Linear discrimination, from the point of view of numerical linear algebra, can be treated as solving an ill-posed system of linear equations. In order to generate a solution that is robust in the presence of noise, these problems require…
Subspace clustering refers to the problem of clustering high-dimensional data that lie in a union of low-dimensional subspaces. State-of-the-art subspace clustering methods are based on the idea of expressing each data point as a linear…
We characterize all possible relative positions between a hyperboloid of one sheet and a sphere through the roots of a characteristic polynomial associated to these quadrics. The classification is also suitable for a hyperboloid and a…
Clustering is essential to many tasks in pattern recognition and computer vision. With the advent of deep learning, there is an increasing interest in learning deep unsupervised representations for clustering analysis. Many works on this…
We study the role of $L_2$ regularization in deep learning, and uncover simple relations between the performance of the model, the $L_2$ coefficient, the learning rate, and the number of training steps. These empirical relations hold when…
The two-dimensional hyperbolic plane, $\mathbb{H}^2$, is an unusual system in that dimensionality changes with scale: locally two-dimensional and planar at short distances, but effectively infinite-dimensional at large scales, it provides…
It is widely accepted in the mode connectivity literature that when two neural networks are trained similarly on the same data, they are connected by a path through parameter space over which test set accuracy is maintained. Under some…
Generalized linear model with $L_1$ and $L_2$ regularization is a widely used technique for solving classification, class probability estimation and regression problems. With the numbers of both features and examples growing rapidly in the…
We consider a high-dimensional mixture of two Gaussians in the noisy regime where even an oracle knowing the centers of the clusters misclassifies a small but finite fraction of the points. We provide a rigorous analysis of the…
In the biology field of botany, leaf shape recognition is an important task. One way of characterising the leaf shape is through the centroid contour distances (CCD). Each CCD path might have different resolution, so normalisation is done…
We show that the topological classification and the smooth classification are generically the same for certain families of plane curves in a semi-local case(the double local case). Especially we give the normal form of transversely jointed…
We classify proper holomorphic mappings between generalized pseudoellipsoids of different dimensions. Those domains are parametrized by the exponents. The relations among them are also obtained. Main tool is the orthogonal decomposition of…
We consider here a generalization of a well known discrete dynamical system produced by the bisection of reflection angles that are constructed recursively between two lines in the Euclidean plane. It is shown that similar properties of…
We study generalization in an overparameterized continual linear regression setting, where a model is trained with L2 (isotropic) regularization across a sequence of tasks. We derive a closed-form expression for the expected generalization…
In the field of machine learning there is a growing interest towards more robust and generalizable algorithms. This is for example important to bridge the gap between the environment in which the training data was collected and the…
The choice of the kernel is critical to the success of many learning algorithms but it is typically left to the user. Instead, the training data can be used to learn the kernel by selecting it out of a given family, such as that of…
We discuss a new renormalization scheme for mixing angles in extended Higgs sectors for the coming era of the Higgs precise measurements at future lepton colliders. We focus on the two Higgs doublet models (2HDMs) with a softly-broken $Z_2$…
The aim of the paper is to introduce an alternative notion of two-scale convergence which gives a more natural modeling approach to the homogenization of partial differential equations with periodically oscillating coefficients: while…