Related papers: Function-Space Optimality of Neural Architectures …

A challenging problem in many modern machine learning tasks is to process weight-space features, i.e., to transform or extract information from the weights and gradients of a neural network. Recent works have developed promising…

This paper delves into the challenging issues in uncertain multi-objective optimization, where uncertainty permeates nonsmooth nonconvex objective and constraint functions. In this context, we investigate highly robust (weakly efficient)…

In this work we provide a characterization of distinct type of (linear and non-linear) maps between Banach spaces in terms of the differentiability of certain class of Lipschitz functions. Our results are stated in an abstract bornological…

Given a training set, a loss function, and a neural network architecture, it is often taken for granted that optimal network parameters exist, and a common practice is to apply available optimization algorithms to search for them. In this…

In recent years, deep neural networks have had great success in machine learning and pattern recognition. Architecture size for a neural network contributes significantly to the success of any neural network. In this study, we optimize the…

We propose a novel framework for solving nonlinear PDEs using sparse radial basis function (RBF) networks. Sparsity-promoting regularization is employed to prevent over-parameterization and reduce redundant features. This work is motivated…

In this paper we study the problem of learning a shallow artificial neural network that best fits a training data set. We study this problem in the over-parameterized regime where the number of observations are fewer than the number of…

We consider the class of spatially decaying systems, where the underlying dynamics are spatially decaying and the sensing and controls are spatially distributed. This class of systems arise in various applications where there is a notion of…

Biological nervous systems consist of networks of diverse, sophisticated information processors in the form of neurons of different classes. In most artificial neural networks (ANNs), neural computation is abstracted to an activation…

We consider composite linear inverse problems where the signal to recover is modeled as a sum of two functions. We study a variational framework formulated as an optimization problem over the pairs of components using two regularization…

Neural Networks (NNs) are the method of choice for building learning algorithms. Their popularity stems from their empirical success on several challenging learning problems. However, most scholars agree that a convincing theoretical…

We introduce the optimality question to the relaxation in multiple control problems described by Sobolev type nonlinear fractional differential equations with nonlocal control conditions in Banach spaces. Moreover, we consider the…

Deep neural networks have emerged as powerful tools for learning operators defined over infinite-dimensional function spaces. However, existing theories frequently encounter difficulties related to dimensionality and limited…

The storage capacity of a binary classification model is the maximum number of random input-output pairs per parameter that the model can learn. It is one of the indicators of the expressive power of machine learning models and is important…

The basic results for nonlinear operators are given. These results include nonlinear versions of classical uniform boundedness theorem and Hahn-Banach theorem. Furthermore, the mappings from a metrizable space into another normed space can…

The foundations of deep learning are supported by the seemingly opposing perspectives of approximation or learning theory. The former advocates for large/expressive models that need not generalize, while the latter considers classes that…

The use of machine learning techniques to improve the performance of branch-and-bound optimization algorithms is a very active area in the context of mixed integer linear problems, but little has been done for non-linear optimization. To…

We study training of Convolutional Neural Networks (CNNs) with ReLU activations and introduce exact convex optimization formulations with a polynomial complexity with respect to the number of data samples, the number of neurons, and data…

We study the least-square regression problem with a two-layer fully-connected neural network, with ReLU activation function, trained by gradient flow. Our first result is a generalization result, that requires no assumptions on the…

Reproducing Kernel Hilbert spaces (RKHS) have been a very successful tool in various areas of machine learning. Recently, Barron spaces have been used to prove bounds on the generalisation error for neural networks. Unfortunately, Barron…