Related papers: Arithmetic Circuits and Neural Networks for Regula…
A significant effort has been made to train neural networks that replicate algorithmic reasoning, but they often fail to learn the abstract concepts underlying these algorithms. This is evidenced by their inability to generalize to data…
The decomposition theory of matroids initiated by Paul Seymour in the 1980's has had an enormous impact on research in matroid theory. This theory, when applied to matrices over the binary field, yields a powerful decomposition theory for…
We present a framework for the construction of linearizations for scalar and matrix polynomials based on dual bases which, in the case of orthogonal polynomials, can be described by the associated recurrence relations. The framework…
We introduce the concept of n-OU and n-OO matrix sets, a collection of n mutually-orthogonal unitary and real orthogonal matrices under Hilbert-Schmidt inner product. We give a detailed characterization of order-three n-OO matrix sets under…
Seymour's Decomposition Theorem for regular matroids states that any matroid representable over both GF(2) and GF(3) can be obtained from matroids that are graphic, cographic, or isomorphic to R10 by 1-, 2-, and 3-sums. It is hoped that…
Elementary symmetric polynomials $S_n^k$ are used as a benchmark for the bounded-depth arithmetic circuit model of computation. In this work we prove that $S_n^k$ modulo composite numbers $m=p_1p_2$ can be computed with much fewer…
Current deep neural networks suffer from two problems; first, they are hard to interpret, and second, they suffer from overfitting. There have been many attempts to define interpretability in neural networks, but they typically lack…
Let $M$ be a matroid defined on a finite set $E$ and $L\subset E$. $L$ is locked in $M$ if $M|L$ and $M^*|(E\backslash L)$ are 2-connected, and $min\{r(L), r^*(E\backslash L)\} \geq 2$. Locked subsets characterize nontrivial facets of the…
A general quantum circuit can be simulated classically in exponential time. If it has a planar layout, then a tensor-network contraction algorithm due to Markov and Shi has a runtime exponential in the square root of its size, or more…
We will consider two special families of polynomial perturbations of the linear center. For the resulting perturbed systems, which are generalized Li\'enard systems, we provide the exact upper bound for the number of limit cycles that…
The width of a neural network matters since increasing the width will necessarily increase the model capacity. However, the performance of a network does not improve linearly with the width and soon gets saturated. In this case, we argue…
We draw connections between simple neural networks and under-determined linear systems to comprehensively explore several interesting theoretical questions in the study of neural networks. First, we emphatically show that it is unsurprising…
We design polynomial size, constant depth (namely, $\mathsf{AC}^0$) arithmetic formulae for the greatest common divisor (GCD) of two polynomials, as well as the related problems of the discriminant, resultant, B\'ezout coefficients,…
This paper aims to interpret the mechanism of feedforward ReLU networks by exploring their solutions for piecewise linear functions, through the deduction from basic rules. The constructed solution should be universal enough to explain some…
The covering radius is a fundamental property of linear codes that characterizes the trade-off between storage and access in linear data-query protocols. The generalized covering radius was recently defined by Elimelech and Schwartz for…
We present new algorithms to detect and correct errors in the lower-upper factorization of a matrix, or the triangular linear system solution, over an arbitrary field. Our main algorithms do not require any additional information or…
Deep neural networks' remarkable ability to correctly fit training data when optimized by gradient-based algorithms is yet to be fully understood. Recent theoretical results explain the convergence for ReLU networks that are wider than…
Successive pairs of pseudo-random numbers generated by standard linear congruential transformations display ordered patterns of parallel lines. We study the ``ordered'' and ``chaotic'' distribution of such pairs by solving the eigenvalue…
ReLU neural networks define piecewise linear functions of their inputs. However, initializing and training a neural network is very different from fitting a linear spline. In this paper, we expand empirically upon previous theoretical work…
We analyze the problem of global reconstruction of functions as accurately as possible, based on partial information in the form of a truncated power series at some point, and additional analyticity properties. This situation occurs…