Related papers: Estimating the energy requirements for long term m…
We show that a protein can be trained to recognise multiple conformations, analogous to an associative memory, and provide capacity calculations based on energy fluctuations and information theory. Unlike the linear capacity of a Hopfield…
We show that in the dissipative quantum model of brain the time-dependence of the frequencies of the electrical dipole wave quanta leads to the dynamical organization of the memories in space (i.e. to their localization in more or less…
Monitoring, understanding, and optimizing the energy consumption of Machine Learning (ML) are various reasons why it is necessary to evaluate the energy usage of ML. However, there exists no universal tool that can answer this question for…
Learning and memory may rely on the ability of neuronal circuits to reorganize by dendritic spine remodeling. We have looked for geometrical parameters of cortical circuits, which maximize information storage capacity associated with this…
The ultimate limits of computation are not just logical, but physical. We investigate the physical resources -- time, energy, entropy, and free energy -- required to perform computational work. We apply the resulting measures of physical…
A generalization of the economic model of natural growth, which takes into account the power-law memory effect, is suggested. The memory effect means the dependence of the process not only on the current state of the process, but also on…
Machine learning (ML) requires using energy to carry out computations during the model training process. The generation of this energy comes with an environmental cost in terms of greenhouse gas emissions, depending on quantity used and the…
Understanding the nature of information storage on memristors is vital to enable their use in novel data storage and neuromorphic applications. One key consideration in information storage is the energy cost of storage and what impact the…
Long range dependence or long memory is a feature of many processes in the natural world, which provides important insights on the underlying mechanisms that generate the observed data. The usual tools available to characterize the…
Memory information in the brain is commonly believed to be stored in the synapse. However, a recent groundbreaking electrophysiology research has raised the possibility that memory information may actually be stored inside the neuron…
Modern machine learning models have started to consume incredible amounts of energy, thus incurring large carbon footprints (Strubell et al., 2019). To address this issue, we have created an energy estimation pipeline1, which allows…
In complex environments, there are costs to both ignorance and perception. An organism needs to track fitness-relevant information about its world, but the more information it tracks, the more resources it must devote to memory and…
The energy efficiency of neuromorphic hardware is greatly affected by the energy of storing, accessing, and updating synaptic parameters. Various methods of memory organisation targeting energy-efficient digital accelerators have been…
We investigate a mutual relationship between information and energy during early phase of LTP induction and maintenance in a large-scale system of mutually coupled dendritic spines, with discrete internal states and probabilistic dynamics,…
Conventional wisdom indicates that initial memory should decay away exponentially in time for general (noncritial) equilibration processes. In particular, time-integrated quantities such as heat are presumed to lose initial memory in a…
It is recently demonstrated that cortical activity can track the time courses of phrases and sentences during speech listening. Here, we propose a plausible neural processing framework to explain this phenomenon. It is argued that the brain…
Transformers are unable to model long-term memories effectively, since the amount of computation they need to perform grows with the context length. While variations of efficient transformers have been proposed, they all have a finite…
The sophisticated tools and techniques employed by Nature for purposeful storage of information stand in stark contrast to the primitive and relatively inefficient means used by man. We describe some impressive features of biological data…
Energy storage shifts energy from off-peak periods to on-peak periods. Unlike conventional generation, storage is duration-limited: the stored energy capacity constrains the duration over which it can supply power. To understand how these…
In-memory deep learning computes neural network models where they are stored, thus avoiding long distance communication between memory and computation units, resulting in considerable savings in energy and time. In-memory deep learning has…