Related papers: On problematic practice of using normalization in …

This paper is devoted to the study of metric subregularity and strong subregularity of any positive order $q$ for set-valued mappings in finite and infinite dimensions. While these notions have been studied and applied earlier for $q=1$…

The phenomenon of implicit regularization has attracted interest in recent years as a fundamental aspect of the remarkable generalizing ability of neural networks. In a nutshell, it entails that gradient descent dynamics in many neural…

The importance and usefulness of renormalization are emphasized in nonrelativistic quantum mechanics. The momentum space treatment of both two-body bound state and scattering problems involving some potentials singular at the origin…

In this article we study the problem of recovering the unknown solution of a linear ill-posed problem, via iterative regularization methods. We review the problem of projection-regularization from a statistical point of view. A basic…

Underpinning the success of deep learning is effective regularizations that allow a variety of priors in data to be modeled. For example, robustness to adversarial perturbations, and correlations between multiple modalities. However, most…

This paper investigates the critical role of eigenalignments between the kernel matrix and learning targets in achieving robust generalization in learning problems. We establish a direct connection between generalization performance in…

While there are many applications of ML to scientific problems that look promising, visuals can be deceiving. Using numerical analysis techniques, we rigorously quantify the accuracy, convergence rates, and generalization bounds of certain…

A systematic mismatch exists between mathematically ideal and effective activation updates during gradient descent. As intended, parameters update in their direction of steepest descent. However, activations are argued to constitute a more…

By the recent advances in computer technology leading to the invention of more powerful processors, the importance of creating models using data training is even greater than ever. Given the significance of this issue, this work tries to…

With model trustworthiness being crucial for sensitive real-world applications, practitioners are putting more and more focus on improving the uncertainty calibration of deep neural networks. Calibration errors are designed to quantify the…

The problem of membrane topology in the matrix model of M-theory is considered. The matrix regularization procedure, which makes a correspondence between finite-sized matrices and functions defined on a two-dimensional base space, is…

Optimization problems are ubiquitous in our societies and are present in almost every segment of the economy. Most of these optimization problems are NP-hard and computationally demanding, often requiring approximate solutions for…

Supervised classification and representation learning are two widely used classes of methods to analyze multivariate images. Although complementary, these methods have been scarcely considered jointly in a hierarchical modeling. In this…

As applied to quantum theories, the program of renormalization is successful for `renormalizable models' but fails for `nonrenormalizable models'. After some conceptual discussion and analysis, an enhanced program of renormalization is…

Neural Networks have been widely used to solve Partial Differential Equations. These methods require to approximate definite integrals using quadrature rules. Here, we illustrate via 1D numerical examples the quadrature problems that may…

Due to the practical importance of vehicle routing problems (VRP), there exists an ever-growing body of research in algorithms and (meta)heuristics for solving such problems. However, the diversity of VRP domains creates the separate…

A recently proposed renormalization scheme can be used to deal with nonrelativistic potential scattering exhibiting ultraviolet divergence in momentum space. A numerical application of this scheme is made in the case of potential scattering…

Decentralized optimization is a powerful paradigm that finds applications in engineering and learning design. This work studies decentralized composite optimization problems with non-smooth regularization terms. Most existing gradient-based…

Diverse inverse problems in imaging can be cast as variational problems composed of a task-specific data fidelity term and a regularization term. In this paper, we propose a novel learnable general-purpose regularizer exploiting recent…

Trained ML models are commonly embedded in optimization problems. In many cases, this leads to large-scale NLPs that are difficult to solve to global optimality. While ML models frequently lead to large problems, they also exhibit…