Related papers: Transient dynamics of associative memory models

Transient amplification has been proposed as an important mechanism not only in neuroscience but in many areas modeled by dynamical systems. Despite that, there is no clear biologically plausible mechanism which fine-tunes the coupling…

We study the dynamical states that emerge in a small-world network of recurrently coupled excitable neurons through both numerical and analytical methods. These dynamics depend in large part on the fraction of long-range connections or…

The co-evolution of structure and dynamics, known as adaptivity, is a fundamental property in various systems and drives diverse emergent behaviors. However, the adaptivity in previous works is primarily stemmed from pairwise situations,…

Trail interactions occur when past particle trajectories bias future motion, rendering the system out of thermodynamic equilibrium. While such systems are abundant in nature, their understanding is limited to the single-particle level or…

The brain can reproduce memories from partial data; this ability is critical for memory recall. The process of memory recall has been studied using auto-associative networks such as the Hopfield model. This kind of model reliably converges…

Deep neural networks with short residual connections have demonstrated remarkable success across domains, but increasing depth often introduces computational redundancy without corresponding improvements in representation quality. We…

Networks with memristive elements (resistors with memory) are being explored for a variety of applications ranging from unconventional computing to models of the brain. However, analytical results that highlight the role of the graph…

Several data-driven approaches based on information theory have been proposed for analyzing high-order interactions involving three or more components of a network system. Most of these methods are defined only in the time domain and rely…

In dynamical systems with distinct time scales the time evolution in phase space may be influenced strongly by the fixed points of the fast subsystem. Orbits then typically follow these points, performing in addition rapid transitions…

Deep learning has been immensely successful at a variety of tasks, ranging from classification to AI. Learning corresponds to fitting training data, which is implemented by descending a very high-dimensional loss function. Understanding…

Transformers encode structure in sequences via an expanding contextual history. However, their purely feedforward architecture fundamentally limits dynamic state tracking. State tracking -- the iterative updating of latent variables…

Many complex networks are known to exhibit sudden transitions between alternative steady states with contrasting properties. Such a sudden transition demonstrates a network's resilience, which is the ability of a system to persist in the…

The neocortex, a complex system driving multi-region interactions, remains a core puzzle in neuroscience. Despite quantitative insights across brain scales, understanding the mechanisms underlying neural activities is challenging. Advances…

An abstract network approach is proposed for the description of the dynamics in reactive processes. The phase space of the variables (concentrations in reactive systems) is partitioned into a finite number of segments, which constitute the…

We introduce and analyze an open quantum generalization of the q-state Potts-Hopfield neural network, which is an associative memory model based on multi-level classical spins. The dynamics of this many-body system is formulated in terms of…

Deep residual architectures, such as ResNet and the Transformer, have enabled models of unprecedented depth, yet a formal understanding of why depth is so effective remains an open question. A popular intuition, following Veit et al.…

We study the recognition capabilities of the Hopfield model with auxiliary hidden layers, which emerge naturally upon a Hubbard-Stratonovich transformation. We show that the recognition capabilities of such a model at zero-temperature…

We investigate the computational limits of the memory retrieval dynamics of modern Hopfield models from the fine-grained complexity analysis. Our key contribution is the characterization of a phase transition behavior in the efficiency of…

We establish mean-field limits for large-scale random-access networks with buffer dynamics and arbitrary interference graphs. While saturated-buffer scenarios have been widely investigated and yield useful throughput estimates for…

We introduce the Dreaming $L$-directional Associative Memory (DLAM), a multi-layer Hebbian architecture in which off-line dreaming and supervised heteroassociative coupling coexist within a single energy function, placing our approach…