Related papers: Fading memory and the convolution theorem
In neuroscience, classical Hopfield networks are the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, memory-augmented neural networks in…
The idea that the vacuum energy density $\rho_{\Lambda}$ could be time dependent is a most reasonable one in the expanding Universe; in fact, much more reasonable than just a rigid cosmological constant for the entire cosmic history. Being…
We pose the converse Madelung question: not whether Fisher information can reproduce quantum mechanics, but whether it is necessary. We work with minimal, physically motivated axioms on density and phase: locality, probability conservation,…
Exploiting data invariances is crucial for efficient learning in both artificial and biological neural circuits. Understanding how neural networks can discover appropriate representations capable of harnessing the underlying symmetries of…
We establish conditions under which a continuous time reservoir computer, such as a leaky integrator echo state network, admits a generalised synchronisation $f$ between between the source dynamics and reservoir dynamics. We show that…
Loss of plasticity is one of the main challenges in continual learning with deep neural networks, where neural networks trained via backpropagation gradually lose their ability to adapt to new tasks and perform significantly worse than…
Recently, many researchers have been focusing on the definition of neural networks for graphs. The basic component for many of these approaches remains the graph convolution idea proposed almost a decade ago. In this paper, we extend this…
Krotov and Hopfield (2021) proposed a biologically plausible two-layer associative memory network with memory storage capacity exponential in the number of visible neurons. However, the capacity was only linear in the number of hidden…
We extend the notion of conservativeness, given by Fredkin and Toffoli in 1982, to generic gates whose input and output lines may assume a finite number d of truth values. A physical interpretation of conservativeness in terms of…
Many learning algorithms have invariances: when their training data is transformed in certain ways, the function they learn transforms in a predictable manner. Here we formalize this notion using concepts from the mathematical field of…
For an upper semi-continuous set-valued mapping from one topological space to another and for a lower semi-continuous function defined on the product of these spaces, Berge's theorem states lower semi-continuity of the minimum of this…
The discussion is limited to first-class parametrized systems, where the definition of time evolution and observables is not trivial, and to finite dimensional systems in order that technicalities do not obscure the conceptual framework.…
Persistence diagrams are efficient descriptors of the topology of a point cloud. As they do not naturally belong to a Hilbert space, standard statistical methods cannot be directly applied to them. Instead, feature maps (or representations)…
A recent breakthrough in biologically-plausible normative frameworks for dimensionality reduction is based upon the similarity matching cost function and the low-rank matrix approximation problem. Despite clear biological interpretation,…
We show that a coherent network of lasers exhibits emergent neural computing capabilities. The proposed scheme is built on harnessing the collective behavior of laser networks for storing a number of phase patterns as stable fixed points of…
We generalize the standard Hopfield model to the case when a weight is assigned to each input pattern. The weight can be interpreted as the frequency of the pattern occurrence at the input of the network. In the framework of the statistical…
We show that the core reasons that complex and hypercomplex valued neural networks offer improvements over their real-valued counterparts is the weight sharing mechanism and treating multidimensional data as a single entity. Their algebra…
Associative memory has been a prominent candidate for the computation performed by the massively recurrent neocortical networks. Attractor networks implementing associative memory have offered mechanistic explanation for many cognitive…
In this paper we investigate how the short-time Fourier transform can be extended in a Clifford setting. We prove some of the main properties of the Clifford short-time Fourier transform such as the orthogonality relation, the…
Structures in low-dimensional topology and low-dimensional geometry -- often combined with ideas from (quantum) field theory -- can explain and inspire concepts in algebra and in representation theory and their categorified versions. We…