Related papers: Categorical Hopfield Networks
Multitask algorithms typically use task similarity information as a bias to speed up and improve the performance of learning processes. Tasks are learned jointly, sharing information across them, in order to construct models more accurate…
Within a fully microscopic setting, we derive a variational principle for the non-equilibrium steady states of chemical reaction networks, valid for time-scales over which chemical potentials can be taken to be slowly varying: at…
In this chapter we survey some particular topics in category theory in a somewhat unconventional manner. Our main focus will be on monoidal categories, mostly symmetric ones, for which we propose a physical interpretation. These are…
The modern Hopfield network, proposed by Krotov and Hopfield, is a mathematical generalization of the Hopfield network, which is a basic model of associative memory that employs higher-order interactions. This study introduces an open…
Differential equations are a ubiquitous tool to study dynamics, ranging from physical systems to complex systems, where a large number of agents interact through a graph with non-trivial topological features. Data-driven approximations of…
We use a coproduct on the time-ordered algebra of field operators to derive simple relations between complete, connected and 1-particle irreducible n-point functions. Compared to traditional functional methods our approach is much more…
The single-layer feedforward neural network with random weights is a recurring motif in the neural networks literature. The advantage of these networks is their simplified training, which reduces to solving a ridge-regression problem. A…
In this paper we study, via equilibrium statistical mechanics, the properties of the internal energy of an Hopfield neural network whose patterns are stored continuously (Gaussian distributed). The model is shown to be equivalent to a…
We introduce a new hierarchical deep learning framework for recursive higher-order meta-learning that enables neural networks (NNs) to construct, solve, and generalise across hierarchies of tasks. Central to this approach is a generative…
Graph neural networks (GNNs) are a powerful tool to learn representations on graphs by iteratively aggregating features from node neighbourhoods. Many variant models have been proposed, but there is limited understanding on both how to…
The Hopfield network model and its generalizations were introduced as a model of associative, or content-addressable, memory. They were widely investigated both as an unsupervised learning method in artificial intelligence and as a model of…
Understanding how people represent categories is a core problem in cognitive science. Decades of research have yielded a variety of formal theories of categories, but validating them with naturalistic stimuli is difficult. The challenge is…
A recent dynamic mean-field theory for sequence processing in fully connected neural networks of Hopfield-type (During, Coolen and Sherrington, 1998) is extended and analized here for a symmetrically diluted network with finite connectivity…
Traditional models of category learning in psychology focus on representation at the category level as opposed to the stimulus level, even though the two are likely to interact. The stimulus representations employed in such models are…
Training deep neural networks (DNNs) can be difficult due to the occurrence of vanishing/exploding gradients during weight optimization. To avoid this problem, we propose a class of DNNs stemming from the time discretization of Hamiltonian…
Hopfield neural networks are a possible basis for modelling associative memory in living organisms. After summarising previous studies in the field, we take a new look at learning rules, exhibiting them as descent-type algorithms for…
Recent advances at the intersection of control theory, neuroscience, and machine learning have revealed novel mechanisms by which dynamical systems perform computation. These advances encompass a wide range of conceptual, mathematical, and…
Network representations of systems from various scientific and societal domains are neither completely random nor fully regular, but instead appear to contain recurring structural building blocks. These features tend to be shared by…
Despite significant advances in graph representation learning, little attention has been paid to the more practical continual learning scenario in which new categories of nodes (e.g., new research areas in citation networks, or new types of…
A categorical framework for modeling and analyzing systems in a broad sense is proposed. These systems should be thought of as `machines' with inputs and outputs, carrying some sort of signal that occurs through some notion of time. Special…