相关论文: Mixed state on a sparsely encoded associative memo…
While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation. To address this, we introduce…
Coded recurrent neural networks with three levels of sparsity are introduced. The first level is related to the size of messages, much smaller than the number of available neurons. The second one is provided by a particular coding rule,…
Diverse cognitive processes set different demands on locally segregated and globally integrated brain activity. However, it remains unclear how resting brains configure their functional organization to balance the demands on network…
The representation of images in the brain is known to be sparse. That is, as neural activity is recorded in a visual area ---for instance the primary visual cortex of primates--- only a few neurons are active at a given time with respect to…
A model of superconducting computer memory exploiting the orthogonal spin transfer (OST) in the pseudospin valve (PS) that is controlled by the three-terminal Josephson superconducting-ferromagnetic transistor (SFT) is developed. The…
The Hopfield model is a pioneering neural network model with associative memory retrieval. The analytical solution of the model in mean field limit revealed that memories can be retrieved without any error up to a finite storage capacity of…
The brain must robustly store a large number of memories, corresponding to the many events encountered over a lifetime. However, the number of memory states in existing neural network models either grows weakly with network size or recall…
After reviewing the behavioral studies of working memory and of the cellular substrate of the latter, we argue that metastable states constitute candidates for the type of transient information storage required by working memory. We then…
In some off-resonant cases, the reduced density matrix of two atoms symmetrically coupled with an optical cavity can very approximately approach to maximally entangled mixed states or maximal Bell violation mixed states in their evolution.…
Sparse representations with learned dictionaries have been successful in several image analysis applications. In this paper, we propose and analyze the framework of ensemble sparse models, and demonstrate their utility in image restoration…
Learning requires the traversal of inherently distinct cognitive states to produce behavioral adaptation. Yet, tools to explicitly measure these states with non-invasive imaging -- and to assess their dynamics during learning -- remain…
Recent advances in large language models highlighted the excessive quadratic cost of self-attention. Despite the significant research efforts, subquadratic attention methods still suffer from inferior performance in practice. We hypothesize…
Contemporary ML separates the static structure of parameters from the dynamic flow of inference, yielding systems that lack the sample efficiency and thermodynamic frugality of biological cognition. In this theoretical work, we propose…
"Sparse" neural networks, in which relatively few neurons or connections are active, are common in both machine learning and neuroscience. Whereas in machine learning, "sparsity" is related to a penalty term that leads to some connecting…
It is well known that the length of a syntactic dependency determines its online memory cost. Thus, the problem of the placement of a head and its dependents (complements or modifiers) that minimizes online memory is equivalent to the…
We consider self-organization and memory formation in a mesoscopic model of an amorphous solid subject to a random shear strain protocol confined to a strain range $\pm \varepsilon_{\rm max}$. We develop proper read-out protocols to show…
An associative memory has been discussed of neural networks consisting of spiking N (=100) Hodgkin-Huxley (HH) neurons with time-delayed couplings, which memorize P patterns in their synaptic weights. In addition to excitatory synapses…
It has been said that complexity lies between order and disorder. In the case of brain activity, and physiology in general, complexity issues are being considered with increased emphasis. We sought to identify features of brain organization…
IntroductionThe free and cued selective reminding test is used to identify memory deficits in mild cognitive impairment and demented patients. It allows assessing three processes: encoding, storage, and recollection of verbal episodic…
We study $k$-uniform states in heterogeneous systems whose local dimensions are mixed. Based on the connections between mixed orthogonal arrays with certain minimum Hamming distance, irredundant mixed orthogonal arrays and $k$-uniform…