Related papers: A Depth Hierarchy for Computing the Maximum in ReL…

In 1989 George Cybenko proved in a landmark paper that wide shallow neural networks can approximate arbitrary continuous functions on a compact set. This universal approximation theorem sparked a lot of follow-up research. Shen, Yang and…

In recent years, functional neural networks have been proposed and studied in order to approximate nonlinear continuous functionals defined on $L^p([-1, 1]^s)$ for integers $s\ge1$ and $1\le p<\infty$. However, their theoretical properties…

In this work, we reveal a rich combinatorial structure underlying exact minimax optimal algorithms for classical nonexpansive fixed-point problems. This viewpoint unifies all extremal optimal methods and provides a systematic and practical…

We prove bounds for the approximation and estimation of certain binary classification functions using ReLU neural networks. Our estimation bounds provide a priori performance guarantees for empirical risk minimization using networks of a…

The $r$-expansion of a $k$-uniform hypergraph $H$, denoted by $H^{(+r)}$, is an $r$-uniform hypergraph obtained by enlarging each $k$-edge of $H$ with a set of $r-k$ vertices of degree one. The random Tur\'an number…

Convex functions and their gradients play a critical role in mathematical imaging, from proximal optimization to Optimal Transport. The successes of deep learning has led many to use learning-based methods, where fixed functions or…

We study graph realization problems from a distributed perspective and we study it in the node capacitated clique (NCC) model of distributed computing, recently introduced for representing peer-to-peer networks. We focus on two central…

This paper studies the approximation capacity of neural networks with an arbitrary activation function and with norm constraint on the weights. Upper and lower bounds on the approximation error of these networks are computed for smooth…

If K is an odd-dimensional flag closed manifold, flag generalized homology sphere or a more general flag weak pseudomanifold with sufficiently many vertices, then the maximal number of edges in K is achieved by the balanced join of cycles.…

Neural networks often operate in the overparameterized regime, in which there are far more parameters than training samples, allowing the training data to be fit perfectly. That is, training the network effectively learns an interpolating…

We study depth separation in infinite-width neural networks, where complexity is controlled by the overall squared $\ell_2$-norm of the weights (sum of squares of all weights in the network). Whereas previous depth separation results…

We give essentially tight bounds for, $\nu(d,k)$, the maximum number of distinct neighbourhoods on a set $X$ of $k$ vertices in a graph with twin-width at most~$d$. Using the celebrated Marcus-Tardos theorem, two independent works [Bonnet…

We develop a novel theoretical framework for analyzing ReLU neural networks through the lens of a combinatorial object we term the ReLU Transition Graph (RTG). In this graph, each node corresponds to a linear region induced by the network's…

We explore convergence of deep neural networks with the popular ReLU activation function, as the depth of the networks tends to infinity. To this end, we introduce the notion of activation domains and activation matrices of a ReLU network.…

In a function approximation with a neural network, an input dataset is mapped to an output index by optimizing the parameters of each hidden-layer unit. For a unary function, we present constraints on the parameters and its second…

Consider the family of all finite graphs with maximum degree $\Delta(G)<d$ and matching number $\nu(G)<m$. In this paper we give a new proof to obtain the exact upper bound for the number of edges in such graphs and also characterize all…

We use deep sparsely connected neural networks to measure the complexity of a function class in $L^2(\mathbb R^d)$ by restricting connectivity and memory requirement for storing the neural networks. We also introduce representation system -…

This paper develops simple feed-forward neural networks that achieve the universal approximation property for all continuous functions with a fixed finite number of neurons. These neural networks are simple because they are designed with a…

It has been shown that deep neural networks of a large enough width are universal approximators but they are not if the width is too small. There were several attempts to characterize the minimum width $w_{\min}$ enabling the universal…

The uniform Tur\'an density $\pi_{u}(F)$ of a $3$-uniform hypergraph (or $3$-graph) $F$ is the supremum of all $d$ such that there exist infinitely many $F$-free $3$-graphs $H$ in which every induced subhypergraph on a linearly sized vertex…