Related papers: The Neuroscience of Transformers
Deep neural networks (DNNs) transform stimuli across multiple processing stages to produce representations that can be used to solve complex tasks, such as object recognition in images. However, a full understanding of how they achieve this…
Understanding the relation between cortical neuronal network structure and neuronal activity is a fundamental unresolved question in neuroscience, with implications to our understanding of the mechanism by which neuronal networks evolve…
Until recently, artificial neural networks were typically designed with a fixed network structure. Here, I argue that network structure is highly relevant to function, and therefore neural networks should be livewired (Eagleman 2020):…
This paper bridges neuroscience and artificial intelligence to propose a cortically inspired blueprint for modular perceptual AI. While current monolithic models such as GPT-4V achieve impressive performance, they often struggle to…
This paper describes some biologically-inspired processes that could be used to build the sort of networks that we associate with the human brain. New to this paper, a 'refined' neuron will be proposed. This is a group of neurons that by…
The paper introduces a biologically and evolutionarily plausible neural architecture that allows a single group of neurons, or an entire cortical pathway, to be dynamically reconfigured to perform multiple, potentially very different…
While modern AI continues to advance, the biological brain remains the pinnacle of neural networks in its robustness, adaptability, and efficiency. This review explores an AI architectural path inspired by the brain's structure,…
Neuroscientists apply a range of common analysis tools to recorded neural activity in order to glean insights into how neural circuits implement computations. Despite the fact that these tools shape the progress of the field as a whole, we…
As our understanding of the mechanisms of brain function is enhanced, the value of insights gained from neuroscience to the development of AI algorithms deserves further consideration. Here, we draw parallels with an existing tree-based ANN…
The deep neural nets of modern artificial intelligence (AI) have not achieved defining features of biological intelligence, including abstraction, causal learning, and energy-efficiency. While scaling to larger models has delivered…
A popular theory of perceptual processing holds that the brain learns both a generative model of the world and a paired recognition model using variational Bayesian inference. Most hypotheses of how the brain might learn these models assume…
While Artificial Neural Networks (ANNs) have yielded impressive results in the realm of simulated intelligent behavior, it is important to remember that they are but sparse approximations of Biological Neural Networks (BNNs). We go beyond…
There is an analogy that is often made between deep neural networks and actual brains, suggested by the nomenclature itself: the "neurons" in deep neural networks should correspond to neurons (or nerve cells, to avoid confusion) in the…
Hierarchies feature prominently in anatomical accounts of cortical organisation. An open question is which computational (algorithmic) processes are implemented by these hierarchies. One renowned hypothesis is that cortical hierarchies…
Over the last decades, cognitive psychology has come to fair consensus about the ontological structure of human intelligence. However, it remains an open question, whether anatomical properties of the brain support the same ontology. The…
Neuroengineering is faced with unique challenges in repairing or replacing complex neural systems that are composed of many interacting parts. These interactions form intricate patterns over large spatiotemporal scales, and produce emergent…
Artificial neural networks have diverged far from their early inspiration in neurology. In spite of their technological and commercial success, they have several shortcomings, most notably the need for a large number of training examples…
The traditional view of neural computation in the cerebral cortex holds that sensory neurons are specialized, i.e., selective for certain dimensions of sensory stimuli. This view was challenged by evidence of contextual interactions between…
Artificial Intelligence (AI) systems based solely on neural networks or symbolic computation present a representational complexity challenge. While minimal representations can produce behavioral outputs like locomotion or simple…
In this paper a novel architecture for cortical computation has been proposed. This architecture is composed of computing paths consisting of neurons and synapses only. These paths have been decomposed into lateral, longitudinal and…