Related papers: Hebbian Crosstalk Prevents Nonlinear Unsupervised …
As an extension of prior work, we study inspecific Hebbian learning using the classical Oja model. We use a combination of analytical tools and numerical simulations to investigate how the effects of inspecificity (or synaptic "cross-talk")…
Purpose: We previously proposed that Hebbian adjustments that are incompletely synapse specific ("crosstalk") might be analogous to genetic mutations. We analyze aspects of the effect of crosstalk in Hebbian learning using the classical Oja…
A central problem in unsupervised deep learning is how to find useful representations of high-dimensional data, sometimes called "disentanglement". Most approaches are heuristic and lack a proper theoretical foundation. In linear…
Continual Learning aims to bring machine learning into a more realistic scenario, where tasks are learned sequentially and the i.i.d. assumption is not preserved. Although this setting is natural for biological systems, it proves very…
It has been demonstrated that one of the most striking features of the nervous system, the so called 'plasticity' (i.e high adaptability at different structural levels) is primarily based on Hebbian learning which is a collection of…
This paper introduces a rate-based nonlinear neural network in which excitatory (E) neurons receive feedforward excitation from sensory (S) neurons, and inhibit each other through disynaptic pathways mediated by inhibitory (I) interneurons.…
Recent work on Long Term Potentiation in brain slices shows that Hebb's rule is not completely synapse-specific, probably due to intersynapse diffusion of calcium or other factors. We extend the classical Oja unsupervised model of learning…
The aim of the present paper is to study the effects of Hebbian learning in random recurrent neural networks with biological connectivity, i.e. sparse connections and separate populations of excitatory and inhibitory neurons. We furthermore…
Hebbian learning is a key principle underlying learning in biological neural networks. We relate a Hebbian spike-timing-dependent plasticity rule to noisy gradient descent with respect to a non-convex loss function on the probability…
The neocortex is widely believed to be the seat of intelligence and "mind". However, it's unclear what "mind" is, or how the special features of neocortex enable it, though likely "connectionist" principles are involved *A. The key to…
Learning in the brain is local and unsupervised (Hebbian). We derive the foundations of an effective human language model inspired by these microscopic constraints. It has two parts: (1) a hierarchy of neurons which learns to tokenize words…
Much has been learned about plasticity of biological synapses from empirical studies. Hebbian plasticity is driven by correlated activity of presynaptic and postsynaptic neurons. Synapses that converge onto the same neuron often behave as…
Cognitive ageing seems to be a story of global degradation. As one ages there are a number of physical, chemical and biological changes that take place. Therefore it is logical to assume that the brain is no exception to this phenomenon.…
Humans can learn several tasks in succession with minimal mutual interference but perform more poorly when trained on multiple tasks at once. The opposite is true for standard deep neural networks. Here, we propose novel computational…
In realistic neural circuits, both neurons and synapses are coupled in dynamics with separate time scales. The circuit functions are intimately related to these coupled dynamics. However, it remains challenging to understand the intrinsic…
In this article we intoduce a novel stochastic Hebb-like learning rule for neural networks that is neurobiologically motivated. This learning rule combines features of unsupervised (Hebbian) and supervised (reinforcement) learning and is…
Adaptation plays a pivotal role in the evolution of natural and artificial complex systems, and in the determination of their functionality. Here, we investigate the impact of adaptive inter-layer processes on intra-layer synchronization in…
The brain performs unsupervised learning and (perhaps) simultaneous supervised learning. This raises the question as to whether a hybrid of supervised and unsupervised methods will produce better learning. Inspired by the rich space of…
We present a mathematical analysis of the effects of Hebbian learning in random recurrent neural networks, with a generic Hebbian learning rule including passive forgetting and different time scales for neuronal activity and learning…
Recent work has shown that biologically plausible Hebbian learning can be integrated with backpropagation learning (backprop), when training deep convolutional neural networks. In particular, it has been shown that Hebbian learning can be…