Related papers: Fading memory and the convolution theorem

It is well known that Convolutional Neural Networks (CNNs) have significant redundancy in their filter weights. Various methods have been proposed in the literature to compress trained CNNs. These include techniques like pruning weights,…

The process of evolutionary diversification unfolds in a vast genotypic space of potential outcomes. During the past century there have been remarkable advances in the development of theory for this diversification, and the theory's success…

We introduce simplicial persistence, a measure of time evolution of network motifs in subsequent temporal layers. We observe long memory in the evolution of structures from correlation filtering, with a two regime power law decay in the…

Persistent Topology studies topological features of shapes by analyzing the lower level sets of suitable functions, called filtering functions, and encoding the arising information in a parameterized version of the Betti numbers, i.e. the…

This work studies the (non)robustness of two-layer neural networks in various high-dimensional linearized regimes. We establish fundamental trade-offs between memorization and robustness, as measured by the Sobolev-seminorm of the model…

Diffusion models power leading generative AI, but when and how they memorize training data, especially on low-dimensional manifolds, remains unclear. We find memorization emerges gradually, not abruptly: as data become scarce, diffusion…

Representation learning on graphs is a fundamental problem that can be crucial in various tasks. Graph neural networks, the dominant approach for graph representation learning, are limited in their representation power. Therefore, it can be…

In [7] Krotov and Hopfield suggest a generalized version of the well-known Hopfield model of associative memory. In their version they consider a polynomial interaction function and claim that this increases the storage capacity of the…

In this work, we explore links between natural homology and persistent homology for the classification of directed spaces. The former is an algebraic invariant of directed spaces, a semantic model of concurrent programs. The latter was…

We consider the bosonic Fock space over the Hilbert space of transversal vector fields in three dimensions. This space carries a canonical representation of the group of rotations. For a certain class of operators in Fock space we show that…

A cut-and-paste model which mimics a trial-and-error process of adaptation is introduced and solved. The model, which can be thought of as a diffusion process with memory, is characterized by two properties, efficiency and persistence. We…

We prove an extension of the theorem of Drinfeld, Grinberg and Kazhdan to arcs with arbitrary residue field. As an application we show that the embedding codimension is generically constant on each irreducible subset of the arc space which…

For an $n$-by-$n$ matrix $A$, let $f_A$ be its "field of values generating function" defined as $f_A\colon x\mapsto x^*Ax$. We consider two natural versions of the continuity, which we call strong and weak, of $f_A^{-1}$ (which is of course…

The driving force behind convolutional networks - the most successful deep learning architecture to date, is their expressive power. Despite its wide acceptance and vast empirical evidence, formal analyses supporting this belief are scarce.…

Successful training of convolutional neural networks is often associated with sufficiently deep architectures composed of high amounts of features. These networks typically rely on a variety of regularization and pruning techniques to…

This expository essay discusses a finite dimensional approach to dilation theory. How much of dilation theory can be worked out within the realm of linear algebra? It turns out that some interesting and simple results can be obtained. These…

Frame theory provides a robust method for recovering vectors in a Hilbert space from inner product data, though the associated decomposition formula can be computationally demanding. We relax the frame condition by studying sequences that…

Convolutional Neural Network (CNN) has been widely used in various fields and played an important role. Convolution operators are the fundamental component of convolutional neural networks, and it is also the most time-consuming part of…

Dependence on the parameter is continuous when perturbations of the parameter preserves strict preference for one alternative over another. We characterise this property via a utility function over alternatives that depends continuously on…

Overparameterized neural networks enjoy great representation power on complex data, and more importantly yield sufficiently smooth output, which is crucial to their generalization and robustness. Most existing function approximation…