Related papers: Density of Neural Network Classes on Compact Subse…

Given a compact manifold $N^n$, an integer $k \in \mathbb{N}_*$ and an exponent $1 \le p < \infty$, we prove that the class $C^\infty(\overline{Q}^m; N^n)$ of smooth maps on the cube with values into $N^n$ is dense with respect to the…

It is an interesting, maybe surprising, fact that different dense subspaces of even "nice" topological spaces can have different densities. So, our aim here is to investigate the set of densities of all dense subspaces of a topological…

In this paper, we study approximation properties of single hidden layer neural networks with weights varying on finitely many directions and thresholds from an open interval. We obtain a necessary and at the same time sufficient measure…

We consider the class of noisy multi-layered sigmoid recurrent neural networks with $w$ (unbounded) weights for classification of sequences of length $T$, where independent noise distributed according to $\mathcal{N}(0,\sigma^2)$ is added…

This paper extends the proof of density of neural networks in the space of continuous (or even measurable) functions on Euclidean spaces to functions on compact sets of probability measures. By doing so the work parallels a more then a…

Neural networks with random weights appear in a variety of machine learning applications, most prominently as the initialization of many deep learning algorithms and as a computationally cheap alternative to fully learned neural networks.…

We show that capacity can be computed with locally Lipschitz functions in locally complete and separable metric spaces. Further, we show that if $(X,d,\mu)$ is a locally complete and separable metric measure space, then continuous functions…

For a topological space X, let (RX)s := (RX,Ts) be the cartesian product of |X| copies of the real line R with the topology of the uniform convergence on separable subsets of X. In this article we analyze the subspace C(X) of (RX)s of all…

Given a topological space $X$, we study the structure of $\infty$-convex subsets in the space $SC_p(X)$ of scatteredly continuous functions on $X$. Our main result says that for a topological space $X$ with countable strong fan tightness,…

We prove that for any topological space $X$ of countable tightness, each \sigma-convex subspace $\F$ of the space $SC_p(X)$ of scatteredly continuous real-valued functions on $X$ has network weight $nw(\F)\le nw(X)$. This implies that for a…

We study regularization in the context of small sample-size learning with over-parameterized neural networks. Specifically, we shift focus from architectural properties, such as norms on the network weights, to properties of the internal…

Proper regularization is critical for speeding up training, improving generalization performance, and learning compact models that are cost efficient. We propose and analyze regularized gradient descent algorithms for learning shallow…

The universal approximation property uniformly with respect to weakly compact families of measures is established for several classes of neural networks. To that end, we prove that these neural networks are dense in Orlicz spaces, thereby…

Deep neural networks use multiple layers of functions to map an object represented by an input vector progressively to different representations, and with sufficient training, eventually to a single score for each class that is the output…

Improvements in the performance of deep neural networks have often come through the design of larger and more complex networks. As a result, fast memory is a significant limiting factor in our ability to improve network performance. One…

Consider a class of probability distributions which is dense in the space of all probability distributions on $\mathbb{R}^{d}$ with respect to weak convergence, for every $d\in\mathbb{N}$. Then, we construct various explicit classes of…

We give characterizations of the bounded subanalytic $\mathscr{C}^\infty$ submanifolds $M$ of $\mathbb{R}^n$ for which the space of Neumann regular functions is dense in Sobolev spaces. By ``Neumann regular function'', we mean a function…

Intuitively, one would expect accuracy of a trained neural network's prediction on test samples to correlate with how densely the samples are surrounded by seen training samples in representation space. We find that a bound on empirical…

Two classes of topological spaces are introduced on which every probability Radon measure possesses a uniformly distributed sequence or a uniformly tight uniformly distributed sequence. It is shown that these classes are stable under…

Even though Deep Neural Networks (DNNs) are widely celebrated for their practical performance, they possess many intriguing properties related to depth that are difficult to explain both theoretically and intuitively. Understanding how…