Related papers: Size of Memory Objects
Feedforward neural networks have been investigated to understand learning and memory, as well as applied to numerous practical problems in pattern classification. It is a rule of thumb that more complex tasks require larger networks.…
A fundamental problem in neuroscience is understanding how working memory -- the ability to store information at intermediate timescales, like 10s of seconds -- is implemented in realistic neuronal networks. The most likely candidate…
We analyze a family of large language models in such a lightweight manner that can be done on a single GPU. Specifically, we focus on the OPT family of models ranging from 125m to 66b parameters and rely only on whether an FFN neuron is…
We use a binary search tree and the simplex algorithm to measure the fraction of patterns that can be stored by an Ising perceptron. The algorithm is much faster than exhaustive search and allows us to obtain accurate statistics up to a…
The human brain contains approximately $10^9$ neurons, each with approximately $10^3$ connections, synapses, with other neurons. Most sensory, cognitive and motor functions of our brains depend on the interaction of a large population of…
A new brain model is introduced, based on the Impulse Pattern Formulation (IPF) already established for modeling and understanding musical instrument and rhythm perception and production. It assumes the brain works with impulses, neural…
Neurons in the nervous system exhibit an outstanding variety of morphological and physiological properties. However, close to threshold, this remarkable richness may be grouped succinctly into two basic types of excitability, often referred…
We propose a computational model of neuron, called firing cell (FC), properties of which cover such phenomena as attenuation of receptors for external stimuli, delay and decay of postsynaptic potentials, modification of internal weights due…
In this work we reveal and explore a new class of attractor neural networks, based on inborn connections provided by model molecular markers, the molecular marker based attractor neural networks (MMBANN). We have explored conditions for the…
Neuronal excitability is the phenomena that describes action potential generation due to a stimulus input. Commonly, neuronal excitability is divided into two classes: Type I and Type II, both having different properties that affect…
Associative memories are devices storing information that can be fully retrieved given partial disclosure of it. We examine a toy model of associative memory and the ultimate limitations it is subjected to within the framework of general…
We introduce and analyze a minimal network model of semantic memory in the human brain. The model is a global associative memory structured as a collection of N local modules, each coding a feature, which can take S possible values, with a…
A population of firing neurons is expected to carry not only mean firing rate but also its fluctuation and synchrony among neurons. In order to examine this possibility, we have studied responses of neuronal ensembles to three kinds of…
This paper extends the Transmission Neural Network model proposed by Gao and Caines in [1]-[3] to incorporate inhibitory connections and neurotransmitter populations. The extended network model contains binary neuronal states, transmission…
A matrix $M: A \times X \rightarrow \{-1,1\}$ corresponds to the following learning problem: An unknown element $x \in X$ is chosen uniformly at random. A learner tries to learn $x$ from a stream of samples, $(a_1, b_1), (a_2, b_2) \ldots$,…
Recent advances in associative memory design through structured pattern sets and graph-based inference algorithms have allowed reliable learning and recall of an exponential number of patterns. Although these designs correct external errors…
Recent generalizations of the Hopfield model of associative memories are able to store a number $P$ of random patterns that grows exponentially with the number $N$ of neurons, $P=\exp(\alpha N)$. Besides the huge storage capacity, another…
In a generic neuron model, we present the linear response theory for the firing rate in response to both time dependent input currents and noise amplitudes. In both cases the signal transmission is strongly attenuated for frequencies above…
Gardner's analysis of the optimal storage capacity of neural networks is extended to study finite-temperature effects. The typical volume of the space of interactions is calculated for strongly-diluted networks as a function of the storage…
We consider the problem of neural association for a network of non-binary neurons. Here, the task is to first memorize a set of patterns using a network of neurons whose states assume values from a finite number of integer levels. Later,…