Related papers: Delving Deeper Into Astromorphic Transformers
Spatial functional organization is a hallmark of biological brains: neurons are arranged topographically according to their response properties, at multiple scales. In contrast, representations within most machine learning models lack…
Neuromorphic engineering is essentially the development of artificial systems, such as electronic analog circuits that employ information representations found in biological nervous systems. Despite being faster and more accurate than the…
Synaptic plasticity is the capacity of a preexisting connection between two neurons to change in strength as a function of neural activity. Because synaptic plasticity is the major candidate mechanism for learning and memory, the…
Transformer-based models have demonstrated exceptional performance across diverse domains, becoming the state-of-the-art solution for addressing sequential machine learning problems. Even though we have a general understanding of the…
The term ``neuromorphic'' refers to systems that are closely resembling the architecture and/or the dynamics of biological neural networks. Typical examples are novel computer chips designed to mimic the architecture of a biological brain,…
Neuromorphic computing has come to refer to a variety of brain-inspired computers, devices, and models that contrast the pervasive von Neumann computer architecture. This biologically inspired approach has created highly connected synthetic…
The thesis explores the role machine learning methods play in creating intuitive computational models of neural processing. Combined with interpretability techniques, machine learning could replace human modeler and shift the focus of human…
Human attention is the gateway to conscious perception, memory and decision-making. However, its role in modern transformer models remains largely unexplored. As these systems increasingly influence what people see, prefer and buy, the…
The quadratic complexity of self-attention mechanism presents a significant impediment to applying Transformer models to long sequences. This work explores computational principles derived from astrocytes-glial cells critical for biological…
The brain consists not only of neurons but also of non-neuronal cells, including astrocytes. Recent discoveries in neuroscience suggest that astrocytes directly regulate neuronal activity by releasing gliotransmitters such as glutamate. In…
A pressing scientific challenge is to understand how brains work. Of particular interest is the neocortex,the part of the brain that is especially large in humans, capable of handling a wide variety of tasks including visual, auditory,…
The dynamics of higher-order topological signals are increasingly recognized as a key aspect of the activity of complex systems. A paradigmatic example are synaptic dynamics: synaptic efficacy changes over time driven by different…
The participation of astrocytes in brain computation was formally hypothesized in 1992, coinciding with the discovery that these glial cells display a complex form of Ca2+ excitability. This fostered conceptual advances centered on the…
A mathematical model of a spiking neuron network accompanied by astrocytes is considered. The network is composed of excitatory and inhibitory neurons with synaptic connections supplied by a memristor-based model of plasticity. Another…
The field of natural language processing has reached breakthroughs with the advent of transformers. They have remained state-of-the-art since then, and there also has been much research in analyzing, interpreting, and evaluating the…
Dynamics play a critical role in computation. The principled evolution of states over time enables both biological and artificial networks to represent and integrate information to make decisions. In the past few decades, significant…
Astrocytes are critical glial cells whose altered morphology and density are hallmarks of many neurological disorders. However, their intricate branching and stain dependent variability make automated detection of histological images a…
Mammalian brains operate in a very special surrounding: to survive they have to react quickly and effectively to the pool of stimuli patterns previously recognized as danger. Many learning tasks often encountered by living organisms involve…
We propose a new mean-field model of brain rhythms governed by astrocytes. This theoretical framework describes how astrocytes can regulate neuronal activity and contribute to the generation of brain rhythms. The model describes at the…
The increasing interest in understanding the behavior of the biological neural networks, and the increasing utilization of artificial neural networks in different fields and scales, both require a thorough understanding of how neuromorphic…