Related papers: On associative neural networks for sparse patterns…
In this work we develop analytical techniques to investigate a broad class of associative neural networks set in the high-storage regime. These techniques translate the original statistical-mechanical problem into an analytical-mechanical…
Neural networks are supposed to recognise blurred images (or patterns) of $N$ pixels (bits) each. Application of the network to an initial blurred version of one of $P$ pre-assigned patterns should converge to the correct pattern. In the…
Sparse coding networks, which utilize unsupervised learning to maximize coding efficiency, have successfully reproduced response properties found in primary visual cortex \cite{AN:OlshausenField96}. However, conventional sparse coding…
Individuals interact and cooperate in structured systems. Many studies represent this structure using static networks, where each link represents a permanent connection between two nodes. However, real interactions are generally not…
There have been many attempts to identify high-dimensional network features via multivariate approaches. Specifically, when the number of voxels or nodes, denoted as p, are substantially larger than the number of images, denoted as n, it…
We introduce a simple cellular-network model to explain the capacity of the plants as memory devices. Following earlier observations (Bose \cite{Bose} and others), we regard the plant as a network in which each of the elements (plant cells)…
Despite the advantageous subquadratic complexity of modern recurrent deep learning models -- such as state-space models (SSMs) -- recent studies have highlighted their potential shortcomings compared to transformers on reasoning and…
We study a model of spiking neurons, with recurrent connections that result from learning a set of spatio-temporal patterns with a spike-timing dependent plasticity rule and a global inhibition. We investigate the ability of the network to…
We introduce a novel approach to endowing neural networks with emergent, long-term, large-scale memory. Distinct from strategies that connect neural networks to external memory banks via intricately crafted controllers and hand-designed…
Automated high-stake decision-making such as medical diagnosis requires models with high interpretability and reliability. As one of the interpretable and reliable models with good prediction ability, we consider Sparse High-order…
Recently, Hopfield and Krotov introduced the concept of {\em dense associative memories} [DAM] (close to spin-glasses with $P$-wise interactions in a disordered statistical mechanical jargon): they proved a number of remarkable features…
Learning or memory formation are associated with the strengthening of the synaptic connections between neurons according to a pattern reflected by the input. According to this theory a retained memory sequence is associated to a dynamic…
Current AI training infrastructure is dominated by single instruction multiple data (SIMD) and systolic array architectures, such as Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs), that excel at accelerating parallel…
The studies of collective oscillations induced by higher-order interactions point out the necessity of group effect in coupling modelization. As yet the related advances are mainly concentrated on nonlinear coupling patterns and cannot be…
The brain is a noisy system subject to energy constraints. These facts are rarely taken into account when modelling artificial neural networks. In this paper, we are interested in demonstrating that those factors can actually lead to the…
Cooperative behaviors are deeply embedded in structured biological and social systems. Networks are often employed to portray pairwise interactions among individuals, where network nodes represent individuals and links indicate who…
Recurrent Neural Networks (RNNs) are used in state-of-the-art models in domains such as speech recognition, machine translation, and language modelling. Sparsity is a technique to reduce compute and memory requirements of deep learning…
The modern Hopfield network, proposed by Krotov and Hopfield, is a mathematical generalization of the Hopfield network, which is a basic model of associative memory that employs higher-order interactions. This study introduces an open…
Cyclically sheared jammed packings form memories of the shear amplitude at which they were trained by falling into periodic orbits where each particle returns to the identical position in subsequent cycles. While simple models that treat…
Complex systems, such as economic, social, biological, and ecological systems, usually feature interactions not only between pairwise entities but also among three or more entities. These multi-entity interactions are known as higher-order…