Related papers: Function-Space Optimality of Neural Architectures …

A linear functional of an object from a convex symmetric set can be optimally estimated, in a worst-case sense, by a linear functional of observations made on the object. This well-known fact is extended here to a nonlinear setting: other…

We provide a convenient framework for the study of the well-posedness of a variety of abstract (integro)differential equations in general Banach function spaces. It allows us to extend and complement the known theory on the maximal…

Lipschitz-constrained neural networks have many applications in machine learning. Since designing and training expressive Lipschitz-constrained networks is very challenging, there is a need for improved methods and a better theoretical…

We study universal approximation of continuous functionals on compact subsets of products of Hilbert spaces. We prove that any such functional can be uniformly approximated by models that first take finitely many continuous linear…

Compared to classical deep neural networks its binarized versions can be useful for applications on resource-limited devices due to their reduction in memory consumption and computational demands. In this work we study deep neural networks…

When optimization theorists consider optimization problems in infinite dimensional spaces, they need to deal with closed convex subsets(usually cones) which mostly have empty interior. These subsets often prevent optimization theorists from…

Despite classical statistical theory predicting severe overfitting, modern massively overparameterized neural networks still generalize well. This unexpected property is attributed to the network's so-called implicit bias, which describes…

This article introduces an innovative mathematical framework designed to tackle non-linear convex variational problems in reflexive Banach spaces. Our approach employs a versatile technique that can handle a broad range of variational…

The paper concerns the investigation of nonconvex and nondifferentiable integral functionals on general Banach spaces, which may not be reflexive and/or separable. Considering two major subdifferentials of variational analysis, we derive…

Why do neurons encode information the way they do? Normative answers to this question model neural activity as the solution to an optimisation problem; for example, the celebrated efficient coding hypothesis frames neural activity as the…

Neural operator architectures approximate operators between infinite-dimensional Banach spaces of functions. They are gaining increased attention in computational science and engineering, due to their potential both to accelerate…

We study the geometry of linear networks with one-dimensional convolutional layers. The function spaces of these networks can be identified with semi-algebraic families of polynomials admitting sparse factorizations. We analyze the impact…

Deep learning has exhibited remarkable results across diverse areas. To understand its success, substantial research has been directed towards its theoretical foundations. Nevertheless, the majority of these studies examine how well deep…

We consider a class of inverse problems defined by a nonlinear map from parameter or model functions to the data. We assume that solutions exist. The space of model functions is a Banach space which is smooth and uniformly convex; however,…

The analysis of neural network training beyond their linearization regime remains an outstanding open question, even in the simplest setup of a single hidden-layer. The limit of infinitely wide networks provides an appealing route forward…

The paper briefy reviews several recent results on hierarchical architectures for learning from examples, that may formally explain the conditions under which Deep Convolutional Neural Networks perform much better in function approximation…

We study Banach-valued holomorphic functions defined on open subsets of the maximal ideal space of the Banach algebra H^\infty of bounded holomorphic functions on the unit disk D\subset C with pointwise multiplication and supremum norm. In…

We present an instructive example of using Banach spaces of solutions to (linear, generally, non-scalar) elliptic operator $A$ to investigate variational inverse problems related to neural networks and/or to regularization of solutions to…

While Bayesian neural networks (BNNs) have drawn increasing attention, their posterior inference remains challenging, due to the high-dimensional and over-parameterized nature. To address this issue, several highly flexible and scalable…

Symmetric functions, which take as input an unordered, fixed-size set, are known to be universally representable by neural networks that enforce permutation invariance. These architectures only give guarantees for fixed input sizes, yet in…