Related papers: Remarks on Feedforward Circuits, Adaptation, and P…
Feedforward CNN models have proven themselves in recent years as state-of-the-art models for predicting single-neuron responses to natural images in early visual cortical neurons. In this paper, we extend these models with recurrent…
Robust control of complex engineered and biological systems hinges on the integration of feedforward and feedback mechanisms. This is exemplified in neural motor control, where feedforward muscle co-contraction complements sensory-driven…
As implied by the plethora of literature on graph rewiring, the choice of computational graph employed by a neural network can make a significant impact on its downstream performance. Certain effects related to the computational graph, such…
Currently, the most successful learning models in computer vision are based on learning successive representations followed by a decision layer. This is usually actualized through feedforward multilayer neural networks, e.g. ConvNets, where…
Feed-forward layers constitute two-thirds of a transformer model's parameters, yet their role in the network remains under-explored. We show that feed-forward layers in transformer-based language models operate as key-value memories, where…
Unlike many physical nonequilibrium systems, in biological systems, the coupling to external energy sources is not a fixed parameter but adaptively controlled by the system itself. We do not have theoretical frameworks that allow for such…
Rhythmic gait patterns in animal locomotion are widely believed to be produced by a central pattern generator (CPG), a network of neurons that drives the muscle groups. In previous papers we have discussed how phase-synchronous signals can…
Landmark universal function approximation results for neural networks with trained weights and biases provided the impetus for the ubiquitous use of neural networks as learning models in neuroscience and Artificial Intelligence (AI). Recent…
Periodic signals propagating along chains are common in biology, for example in locomotion and peristalsis, and are also of interest for continuum robots. In previous work we constructed such networks as 'feedforward lifts' of a central…
Matching animal-like flexibility in recognition and the ability to quickly incorporate new information remains difficult. Limits are yet to be adequately addressed in neural models and recognition algorithms. This work proposes a…
For nonlinear systems that are known to be globally asymptotically stabilizable, control over networks introduces a major challenge because of the asynchrony in the transmission schedule. Maintaining global asymptotic stabilization in…
A model of an organism as an autonomous intelligent system has been proposed. This model was used to analyze learning of an organism in various environmental conditions. Processes of learning were divided into two types: strong and weak…
In this work, it is demonstrated that the usual power system dynamic model exhibits a feedforward-feedback control structure. The distinct properties of the feedforward and feedback subsystems are identified and studied using respective…
The importance of pulse-coupled oscillators (PCOs) in biology and engineering has motivated research to understand basic properties of PCO networks. Despite the large body of work addressing PCOs, a global synchronization result for…
Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically,…
Associative learning is one of the key mechanisms displayed by living organisms in order to adapt to their changing environments. It was early recognized to be a general trait of complex multicellular organisms but also found in "simpler"…
This paper deals with the trajectory tracking control problem for a class of bilinear systems with unmeasurable states and unknown parameters. Firstly, a full-information controller is suggested that guarantees global tracking under a…
Directed cycles form the fundamental motifs in natural, social and artificial networks, yet their distinct computational roles remain under-explored, particularly in the context of higher-order structure and function. In this work, we…
Habituation - a phenomenon in which a dynamical system exhibits a diminishing response to repeated stimulations that eventually recovers when the stimulus is withheld - is universally observed in living systems from animals to unicellular…
Theoretical efforts to prove advantages of Transformers in comparison with classical architectures such as feedforward and recurrent neural networks have mostly focused on representational power. In this work, we take an alternative…