Related papers: Finite Connectivity Attractor Neural Networks
Glass networks model systems of variables that interact via sharp switching. A body of theory has been developed over several decades that, in principle, allows rigorous proof of dynamical properties in high dimensions that is not normally…
Artificial intelligence based on artificial neural networks, which are originally inspired by the biological architectures of human brain, has mostly been realized using software but executed on conventional von Neumann computers, where the…
We study Ising spin models on finitely connected random interaction graphs which are drawn from an ensemble in which not only the degree distribution $p(k)$ can be chosen arbitrarily, but which allows for further fine-tuning of the topology…
The brain is targeted for processing temporal sequence information. It remains largely unclear how the brain learns to store and retrieve sequence memories. Here, we study how recurrent networks of binary neurons learn sequence attractors…
By using real space renormalisation group (RG) methods we show that spin-glasses in a field display a new kind of transition in high dimensions. The corresponding critical properties and the spin-glass phase are governed by two…
In traditional recognition tasks of neural networks, a potential landscape or cost function guides the system toward patterns using gradient dynamics. That is not how the brain works as its dynamics is far from equilibrium. We present an…
Over the past few years, neural network methods have evolved in various directions for approximating partial differential equations (PDEs). A promising new development is the integration of neural networks with classical numerical…
Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that…
We propose an approach that connects recurrent networks with different orders of hidden interaction with regular grammars of different levels of complexity. We argue that the correspondence between recurrent networks and formal…
A recurrent neural network is considered that can retrieve a collection of patterns, as well as slightly perturbed versions of this `pure' set of patterns via fixed points of its dynamics. By replacing the set of dynamical constraints,…
We include p-spin interactions in a spherical version of a soluble mean-field spin-glass model proposed by van Hemmen. Due to the simplicity of the solutions, which do not require the use of the replica trick, we are able to carry out a…
We study the topological properties of attractors of Iterated Function Systems (I.F.S.) on the real line, consisting of affine maps of homogeneous contraction ratio. These maps define what we call a second generation I.F.S.: they are…
We introduce a diluted version of the one dimensional spin-glass model with interactions decaying in probability as an inverse power of the distance. In this model varying the power corresponds to change the dimension in short-range models.…
Residual deep neural networks (ResNets) are mathematically described as interacting particle systems. In the case of infinitely many layers the ResNet leads to a system of coupled system of ordinary differential equations known as neural…
We study associative memory neural networks of the Hodgkin-Huxley type of spiking neurons in which multiple periodic spatio-temporal patterns of spike timing are memorized as limit-cycle-type attractors. In encoding the spatio-temporal…
Inner products of neural network feature maps arise in a wide variety of machine learning frameworks as a method of modeling relations between inputs. This work studies the approximation properties of inner products of neural networks. It…
Exact solutions for the learning problem of autoassociative networks with binary couplings are determined by a new method. The use of a branch-and-bound algorithm leads to a substantial saving of computational time compared with complete…
At the mean-field level, on fully connected lattices, several disordered spin models have been shown to belong to the universality class of "structural glasses", with a "random first-order transition" (RFOT) characterized by a discontinuous…
In this paper, we propose several models, which can realize synchronization of complex networks in finite time effectively. The results apply to heterogeneous dynamic networks, too. The mechanism of finite time convergence is revealed.…
Recent results are reviewed on both the time evolution and retrieval properties of multi-state neural networks that are based upon spin-glass models. In particular, the properties of models with neuron states having Q-Ising symmetry are…