Simultaneous embedding of multiple attractor manifolds in a recurrent neural network using constrained gradient optimization
Abstract
The storage of continuous variables in working memory is hypothesized to be sustained in the brain by the dynamics of recurrent neural networks (RNNs) whose steady states form continuous manifolds. In some cases, it is thought that the synaptic connectivity supports multiple attractor manifolds, each mapped to a different context or task. For example, in hippocampal area CA3, positions in distinct environments are represented by distinct sets of population activity patterns, each forming a continuum. It has been argued that the embedding of multiple continuous attractors in a single RNN inevitably causes detrimental interference: quenched noise in the synaptic connectivity disrupts the continuity of each attractor, replacing it by a discrete set of steady states that can be conceptualized as lying on local minima of an abstract energy landscape. Consequently, population activity patterns exhibit systematic drifts towards one of these discrete minima, thereby degrading the stored memory over time. Here we show that it is possible to dramatically attenuate these detrimental interference effects by adjusting the synaptic weights. Synaptic weight adjustments are derived from a loss function that quantifies the roughness of the energy landscape along each of the embedded attractor manifolds. By minimizing this loss function, the stability of states can be dramatically improved, without compromising the capacity.
Cite
@article{arxiv.2310.18708,
title = {Simultaneous embedding of multiple attractor manifolds in a recurrent neural network using constrained gradient optimization},
author = {Haggai Agmon and Yoram Burak},
journal= {arXiv preprint arXiv:2310.18708},
year = {2023}
}
Comments
To be presetned at the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)