Related papers: Harmonic Decompositions of Convolutional Networks
We consider the problem of learning a set from random samples. We show how relevant geometric and topological properties of a set can be studied analytically using concepts from the theory of reproducing kernel Hilbert spaces. A new kind of…
In this article we recover the distribution function (and possible density) of an arbitrary random variable that is subject to an additive measurement error. This problem is also known as deconvolution and has a long tradition in…
This paper addresses the problem of regression to reconstruct functions, which are observed with superimposed errors at random locations. We address the problem in reproducing kernel Hilbert spaces. It is demonstrated that the estimator,…
Symmetries impose structure on the Hilbert space of a quantum mechanical model. The mathematical units of this structure are the irreducible representations of symmetry groups and I consider how they function as conceptual units of…
Symmetric functions, which take as input an unordered, fixed-size set, are known to be universally representable by neural networks that enforce permutation invariance. These architectures only give guarantees for fixed input sizes, yet in…
Normalization layers and activation functions are fundamental components in deep networks and typically co-locate with each other. Here we propose to design them using an automated approach. Instead of designing them separately, we unify…
In this work, we consider "finite bandwidth" reproducing kernel Hilbert spaces which have orthonormal bases of the form $f_n(z)=z^n \prod_{j=1}^J \left( 1 - a_{n}w_j z \right)$, where $w_1 ,w_2, \ldots w_J $ are distinct points on the…
Convolutional Neural Networks (CNNs) are known to be significantly over-parametrized, and difficult to interpret, train and adapt. In this paper, we introduce a structural regularization across convolutional kernels in a CNN. In our…
While the universal approximation property holds both for hierarchical and shallow networks, we prove that deep (hierarchical) networks can approximate the class of compositional functions with the same accuracy as shallow networks but with…
We develop sampling formulas for high-dimensional functions in reproducing kernel Hilbert spaces, where we rely on irregular samples that are taken at determining sequences of data points. We place particular emphasis on sampling formulas…
The theory of positive kernels and associated reproducing kernel Hilbert spaces, especially in the setting of holomorphic functions, has been an important tool for the last several decades in a number of areas of complex analysis and…
The convolution operation is a central building block of neural network architectures widely used in computer vision. The size of the convolution kernels determines both the expressiveness of convolutional neural networks (CNN), as well as…
Treating neural network inputs and outputs as random variables, we characterize the structure of neural networks that can be used to model data that are invariant or equivariant under the action of a compact group. Much recent research has…
Traditional machine learning models, particularly neural networks, are rooted in finite-dimensional parameter spaces and nonlinear function approximations. This report explores an alternative formulation where learning tasks are expressed…
We present decompositions of various positive kernels as integrals or sums of positive kernels. Within this framework we study the reproducing kernel Hilbert spaces associated with the fractional and bi-fractional Brownian motions. As a…
Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks as the curse of dimensionality (CoD) cannot be evaded when trying to approximate even a single ReLU neuron…
In this work, we consider the problem of learning nonlinear operators that correspond to discrete-time nonlinear dynamical systems with inputs. Given an initial state and a finite input trajectory, such operators yield a finite output…
In this paper we investigate the ability of a neural network to approximate algebraic properties associated to lattice simplices. In particular we attempt to predict the distribution of Hilbert basis elements in the fundamental…
Recent work has shown that parameterizing and optimizing coordinate transformations using normalizing flows, i.e., invertible neural networks, can significantly accelerate the convergence of spectral approximations. We present the first…
We establish a regular sampling theory in the range of the analysis operator of a continuous frame having a unitary structure. The unitary structure is related with a unitary representation of a locally compact abelian group on a separable…