Related papers: Neural networks with high order connections

High order networks are weighted hypergraphs col- lecting relationships between elements of tuples, not necessarily pairs. Valid metric distances between high order networks have been defined but they are difficult to compute when the…

Network design has been a central topic in machine learning. Large amounts of effort have been devoted towards creating efficient architectures through manual exploration as well as automated neural architecture search. However, todays…

Learning a similarity metric has gained much attention recently, where the goal is to learn a function that maps input patterns to a target space while preserving the semantic distance in the input space. While most related work focused on…

We consider a three-layer Sejnowski machine and show that features learnt via contrastive divergence have a dual representation as patterns in a dense associative memory of order P=4. The latter is known to be able to Hebbian-store an…

Gardner's analysis of the optimal storage capacity of neural networks is extended to study finite-temperature effects. The typical volume of the space of interactions is calculated for strongly-diluted networks as a function of the storage…

Several data-driven approaches based on information theory have been proposed for analyzing high-order interactions involving three or more components of a network system. Most of these methods are defined only in the time domain and rely…

The current study uses a novel method of multilevel neurons and high order synchronization effects described by a family of special metrics, for pattern recognition in an oscillatory neural network (ONN). The output oscillator (neuron) of…

Ever since the last two decades of the past century pioneering studies in the field of statistical physics had focused their efforts on developing models of neural networks that could display memory storage and retrieval. Though many…

We investigate the maximum number of embedded patterns in the two-dimensional Hopfield model. The grand state energies of two specific network states, namely, the energies of the pure-ferromagnetic state and the state of specific one stored…

There has been recent progress on the problem of inferring the structure of interactions in complex networks when they are in stationary states satisfying detailed balance, but little has been done for non-equilibrium systems. Here we…

Describing the collective activity of neural populations is a daunting task: the number of possible patterns grows exponentially with the number of cells, resulting in practically unlimited complexity. Recent empirical studies, however,…

We study the synchronous dynamics of the Hopfield model when a random antisymmetric part is added to the otherwise symmetric synaptic matrix. We use a generating functional technique to derive analytical expressions for the order parameters…

We investigate the computational limits of the memory retrieval dynamics of modern Hopfield models from the fine-grained complexity analysis. Our key contribution is the characterization of a phase transition behavior in the efficiency of…

The ability of discrete-time nonlinear recurrent neural networks to store time-varying small input signals is investigated by mean-field theory. The combination of a small input strength and mean-field assumptions makes it possible to…

In neuroscience, classical Hopfield networks are the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, memory-augmented neural networks in…

Circuits of biological neurons, such as in the functional parts of the brain can be modeled as networks of coupled oscillators. Inspired by the ability of these systems to express a rich set of outputs while keeping (gradients of) state…

The Hopfield associative memory model stores random patterns in synaptic couplings according to Hebb's rule and retrieves them through gradient descent on an energy function. This conventional setting, where neurons are assumed to have…

Stack-augmented recurrent neural networks (RNNs) have been of interest to the deep learning community for some time. However, the difficulty of training memory models remains a problem obstructing the widespread use of such models. In this…

Cyclic patterns of neuronal activity are ubiquitous in animal nervous systems, and partially responsible for generating and controlling rhythmic movements such as locomotion, respiration, swallowing and so on. Clarifying the role of the…

Higher-order information is crucial for relational learning in many domains where relationships extend beyond pairwise interactions. Hypergraphs provide a natural framework for modeling such relationships, which has motivated recent…