Related papers: Associative memory and dead neurons
Recent research has established a connection between modern Hopfield networks (HNs) and transformer attention heads, with guarantees of exponential storage capacity. However, these models still face challenges scaling storage efficiently.…
The impairment of cognitive function in Alzheimer's is clearly correlated to synapse loss. However, the mechanisms underlying this correlation are only poorly understood. Here, we investigate how the loss of excitatory synapses in sparsely…
A memristor is a nonlinear two-terminal electrical element that incorporates memory features and nanoscale properties, enabling us to design very high-density artificial neural networks. To enhance the memory property, we should use…
The stability and convergence of the neural networks are the fundamental characteristics in the Hopfield type networks. Since time delay is ubiquitous in most physical and biological systems, more attention is being made for the delayed…
Estimating the region of attraction (ROA) of general nonlinear autonomous systems remains a challenging problem and requires a case-by-case analysis. Leveraging the universal approximation property of neural networks, in this paper, we…
In recent years there has been a growing interest in the physical implementation of classical spin models through networks of optical oscillators. However, a key missing step in this mapping is to formally prove that the dynamics of such a…
Fixed-time stable dynamical systems are capable of achieving exact convergence to an equilibrium point within a fixed time that is independent of the initial conditions of the system. This property makes them highly appealing for designing…
In this chapter, we utilize dynamical systems to analyze several aspects of machine learning algorithms. As an expository contribution we demonstrate how to re-formulate a wide variety of challenges from deep neural networks, (stochastic)…
We study analytically and numerically a Hopfield fully-connected network with $d$-dimensional vector spins. These networks are models of associative memory that generalize the standard Hopfield with Ising spins, where $P$ examples are…
Collective rhythmic dynamics from neurons is vital for cognitive functions such as memory formation but how neurons self-organize to produce such activity is not well understood. Attractor-based models have been successfully implemented as…
We consider the problem of nonlinear system identification when prior knowledge is available on the region of attraction (ROA) of an equilibrium point. We propose an identification method in the form of an optimization problem, minimizing…
The Hopfield recurrent neural network is a classical auto-associative model of memory, in which collections of symmetrically-coupled McCulloch-Pitts neurons interact to perform emergent computation. Although previous researchers have…
An approach to non-adiabatic dynamics of atoms in molecular and condensed matter systems under general non-equilibrium conditions is proposed. In this method interaction between nuclei and electrons is considered explicitly up to the second…
Learning for control of dynamical systems with formal guarantees remains a challenging task. This paper proposes a learning framework to simultaneously stabilize an unknown nonlinear system with a neural controller and learn a neural…
We show that memory, feedback, and activity are all describable by the same unifying concept, that is non-reciprocal (NR) coupling. We demonstrate that characteristic thermodynamic features of these intrinsically nonequilibrium systems are…
Stochastic network-dynamics are typically assumed to be memory-less. Involving prolonged dwells interrupted by instantaneous transitions between nodes such Markov networks stand as a coarse-graining paradigm for chemical reactions, gene…
Understanding the dynamics of large-scale brain models remains a central challenge due to the inherent complexity of these systems. In this work, we explore the emergence of complex spatiotemporal patterns in a large scale-brain model…
The solution of complex many-body lattice models can often be found by defining an energy functional of the relevant density of the problem. For instance, in the case of the Hubbard model the spin-resolved site occupation is enough to…
We propose and analyse a procedure for using a standard activity-based neuron network model and firing data to compute the effective connection strengths between neurons in a network. We assume a Heaviside response function, that the…
Normal mode analysis (NMA) provides a mathematical framework for exploring the intrinsic global dynamics of molecules through the definition of an energy function, where normal modes correspond to the eigenvectors of the Hessian matrix…