Related papers: Parametrical Neural Network
Graph neural networks (GNNs) are a class of deep models that operate on data with arbitrary topology represented as graphs. We introduce an efficient memory layer for GNNs that can jointly learn node representations and coarsen the graph.…
In this work we study numerically the out of equilibrium dynamics of the Hopfield model for associative memory inside its spin-glass phase. Besides its interest as a neural network model it can also be considered as a prototype of fully…
This paper investigates deep neural network (DNN) compression from the perspective of compactly representing and storing trained parameters. We explore the previously overlooked opportunity of cross-layer architecture-agnostic…
A fiber optic channel is modeled in a variety of ways; from the simple additive white complex Gaussian noise model, to models that incorporate memory in the channel. Because of Kerr nonlinearity, a simple model is not a good approximation…
We show that a protein can be trained to recognise multiple conformations, analogous to an associative memory, and provide capacity calculations based on energy fluctuations and information theory. Unlike the linear capacity of a Hopfield…
It is known that $O(N)$ parameters are sufficient for neural networks to memorize arbitrary $N$ input-label pairs. By exploiting depth, we show that $O(N^{2/3})$ parameters suffice to memorize $N$ pairs, under a mild condition on the…
Neural network pruning techniques reduce the number of parameters without compromising predicting ability of a network. Many algorithms have been developed for pruning both over-parameterized fully-connected networks (FCNs) and…
We examine numerically the storage capacity and the behaviour near saturation of an attractor neural network consisting of bistable elements with an adjustable coupling strength, the Bistable Gradient Network (BGN). For strong coupling, we…
It is believed that neural information representation and processing relies on the neural population instead of a single neuron. In neuromorphic photonics, photonic neurons in the form of nonlinear responses have been extensively studied in…
We develop a theory of percolation with plasticity systems (PWPs) rendering properties of interest for neuromorphic computing. Unlike the standard percolation between two large electrodes, they have multiple ($N\gg 1$) interfaces and…
We use belief-propagation techniques to study the equilibrium behavior of a bipartite spin-glass, with interactions between two sets of $N$ and $P = \alpha N$ spins. Each spin has a finite degree, i.e.\ number of interaction partners in the…
The empirical results suggest that the learnability of a neural network is directly related to its size. To mathematically prove this, we borrow a tool in topological algebra: Betti numbers to measure the topological geometric complexity of…
Many of today's most interesting questions involve understanding and interpreting complex relationships within graph-based structures. For instance, in materials science, predicting material properties often relies on analyzing the…
Neural networks are supposed to recognise blurred images (or patterns) of $N$ pixels (bits) each. Application of the network to an initial blurred version of one of $P$ pre-assigned patterns should converge to the correct pattern. In the…
Recent studies point to the potential storage of a large number of patterns in the celebrated Hopfield associative memory model, well beyond the limits obtained previously. We investigate the properties of new fixed points to discover that…
This paper considers the problem of information capacity of a random neural network. The network is represented by matrices that are square and symmetrical. The matrices have a weight which determines the highest and lowest possible value…
Associative memory and probabilistic modeling are two fundamental topics in artificial intelligence. The first studies recurrent neural networks designed to denoise, complete and retrieve data, whereas the second studies learning and…
Neural associative memories are single layer perceptrons with fast synaptic learning typically storing discrete associations between pairs of neural activity patterns. Previous works have analyzed the optimal networks under naive Bayes…
In this paper, we present a new kind of learning implementation to recognize the patterns using the concept of Mirroring Neural Network (MNN) which can extract information from distinct sensory input patterns and perform pattern recognition…
Associative Memories like the famous Hopfield Networks are elegant models for describing fully recurrent neural networks whose fundamental job is to store and retrieve information. In the past few years they experienced a surge of interest…