Related papers: On the structure of multi-layer cellular neural ne…

A Hadamard-Hitchcock decomposition of a multidimensional array is a decomposition that expresses the latter as a Hadamard product of several tensor rank decompositions. Such decompositions can encode probability distributions that arise…

Depth separation -- why a deeper network is more powerful than a shallower one -- has been a major problem in deep learning theory. Previous results often focus on representation power. For example, arXiv:1904.06984 constructed a function…

We investigate the computational complexity of computing the Hausdorff distance. Specifically, we show that the decision problem of whether the Hausdorff distance of two semi-algebraic sets is bounded by a given threshold is complete for…

Let $A$ be a set and $V$ a real Hilbert space. Let $H$ be a real Hilbert space of functions $f:A\to V$ and assume $H$ is continuously embedded in the Banach space of bounded functions. For $i=1,\cdots,n$, let $(x_i,y_i)\in A\times V$…

We introduce and study the \emph{Lattice Distortion Problem} (LDP). LDP asks how "similar" two lattices are. I.e., what is the minimal distortion of a linear bijection between the two lattices? LDP generalizes the Lattice Isomorphism…

For a tree $T$, we show that for many positive integer values of $n$, and an integer $s \geq 2$, the higher topological complexity $TC_s$ of the unordered configuration spaces of trees $U\mathcal{C}^nT$, is maximal. In other words, we prove…

We developed a tool for visualizing and analyzing large pre-trained vision models by mapping them onto the brain, thus exposing their hidden inside. Our innovation arises from a surprising usage of brain encoding: predicting brain fMRI…

Finding hidden layers in complex networks is an important and a non-trivial problem in modern science. We explore the framework of quantum graphs to determine whether concealed parts of a multi-layer system exist and if so then what is…

Deep neural networks can be effective means to automatically classify aerial images but is easy to overfit to the training data. It is critical for trained neural networks to be robust to variations that exist between training and test…

Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has…

This work investigates minimal parametric networks in hyperspaces of closed subsets of metric spaces endowed with the Hausdorff distance. It is shown that the problems of finding such networks are nontrivial only within finiteness classes,…

We show that the core reasons that complex and hypercomplex valued neural networks offer improvements over their real-valued counterparts is the weight sharing mechanism and treating multidimensional data as a single entity. Their algebra…

It has long been conjectured that hypotheses spaces suitable for data that is compositional in nature, such as text or images, may be more efficiently represented with deep hierarchical networks than with shallow ones. Despite the vast…

Deep learning techniques have demonstrated significant capacity in modeling some of the most challenging real world problems of high complexity. Despite the popularity of deep models, we still strive to better understand the underlying…

Despite the remarkable success of large large-scale neural networks, we still lack unified notation for thinking about and describing their representational spaces. We lack methods to reliably describe how their representations are…

Let $F$ be a field. We show that certain subrings contained between the polynomial ring $F[X] = F[X_1, ..., X_n]$ and the power series ring $F[X][[Y]] = F[X_1, ..., X_n][[Y]]$ have Weierstrass Factorization, which allows us to deduce both…

Let $E \subseteq R^n$ be a closed set of Hausdorff dimension $\alpha$. For $m \geq n$, let $\{B_1,\ldots,B_k\}$ be $n \times (m-n)$ matrices. We prove that if the system of matrices $B_j$ is non-degenerate in a suitable sense, $\alpha$ is…

Multilayer networks allow for modeling complex relationships, where individuals are embedded in multiple social networks at the same time. Given the ubiquity of such relationships, these networks have been increasingly gaining attention in…

We address the structure identification and the uniform approximation of two fully nonlinear layer neural networks of the type $f(x)=1^T h(B^T g(A^T x))$ on $\mathbb R^d$ from a small number of query samples. We approach the problem by…

In this paper, we measure the linear separability of hidden layer outputs to study the characteristics of deep neural networks. In particular, we first propose Minkowski difference based linear separability measures (MD-LSMs) to evaluate…