Related papers: Understanding neural networks with reproducing ker…
Neural networks are a powerful class of functions that can be trained with simple gradient descent to achieve state-of-the-art performance on a variety of applications. Despite their practical success, there is a paucity of results that…
The empirical success of deep convolutional networks on tasks involving high-dimensional data such as images or audio suggests that they can efficiently approximate certain functions that are well-suited for such tasks. In this paper, we…
In continuation of the paper [3], we discuss various consequences of Hahn-Banach theorem for bounded b-linear functional in linear n-normed space and describe the notion of reflexivity of linear n-normed space with respect to bounded…
Deep ReLU Networks can be decomposed into a collection of linear models, each defined in a region of a partition of the input space. This paper provides three results extending this theory. First, we extend this linear decompositions to…
A Bochner integral formula is derived that represents a function in terms of weights and a parametrized family of functions. Comparison is made to pointwise formulations, norm inequalities relating pointwise and Bochner integrals are…
This paper explores some important aspects of the theory of rearrangement-invariant quasi-Banach function spaces. We focus on two main topics. Firstly, we prove an analogue of the Luxemburg representation theorem for rearrangement-invariant…
The Lizorkin space is well-suited for studying various operators; e.g., fractional Laplacians and the Radon transform. In this paper, we show that the space is unfortunately not complemented in the Schwartz space. However, we can show that…
In this paper, we leverage a recent deep kernel representer theorem to connect kernel based learning and (deep) neural networks in order to understand their interplay. In particular, we show that the use of special types of kernels yields…
Motivated by the abundance of functional data such as time series and images, there has been a growing interest in integrating such data into neural networks and learning maps from function spaces to R (i.e., functionals). In this paper, we…
If a Banach-space operator has a complemented range, then its normed-space adjoint has a complemented kernel and the converse holds on a reflexive Banach space. It is also shown when complemented kernel for an operator is equivalent to…
Previous work has questioned the conditions under which the decision regions of a neural network are connected and further showed the implications of the corresponding theory to the problem of adversarial manipulation of classifiers. It has…
All most all the function spaces over real or complex domains and spaces of sequences, that arise in practice as examples of normed complete linear spaces (Banach spaces), are reflexive. These Banach spaces are dual to their respective…
We present a construction that enables one to find Banach spaces $X$ whose sets $NA(X)$ of norm attaining functionals do not contain two-dimensional subspaces and such that, consequently, $X$ does not contain proximinal subspaces of finite…
It is well known that in the calculus of variations and in optimization there exist many formulations of the fundamental propositions on the attainment of the infima of sequentially weakly lower semicontinuous coercive functions on…
In this paper, we show that, under mild assumptions, input-output behavior of a continous-time recurrent neural network (RNN) can be represented by a rational or polynomial nonlinear system. The assumptions concern the activation function…
We develop a corrective mechanism for neural network approximation: the total available non-linear units are divided into multiple groups and the first group approximates the function under consideration, the second group approximates the…
We analyse the convergence of sampling algorithms for functions in reproducing kernel Hilbert spaces (RKHS). To this end, we discuss approximation properties of kernel regression under minimalistic assumptions on both the kernel and the…
We provide the first mathematically complete derivation of the Nystr\"om method for low-rank approximation of indefinite kernels and propose an efficient method for finding an approximate eigendecomposition of such kernel matrices. Building…
We perform a study on the generalization ability of the wide two-layer ReLU neural network on $\mathbb{R}$. We first establish some spectral properties of the neural tangent kernel (NTK): $a)$ $K_{d}$, the NTK defined on $\mathbb{R}^{d}$,…
In a previous paper, we have shown that a recurrent neural network (RNN) can be used to detect cellular network radio signal degradations accurately. We unexpectedly found, though, that accuracy gains diminished as we added layers to the…