Related papers: Feedforward Architectures Driven by Inhibitory Int…

The human organism is an integrated network where complex physiologic systems, each with its own regulatory mechanisms, continuously interact, and where failure of one system can trigger a breakdown of the entire network. Identifying and…

Multi-agent systems in biology, society, and engineering are capable of making decisions through the dynamic interaction of their elements. Nonlinearity of the interactions is key for the speed, robustness, and flexibility of multi-agent…

Network structure plays a critical role in functionality and performance of network systems. This paper examines structural adaptivity of diffusively coupled, directed multi-agent networks that are subject to diffusion performance. Inspired…

Directed networks are essential for representing complex systems, capturing the asymmetry of interactions in fields such as neuroscience, transportation, and social networks. Directionality reveals how influence, information, or resources…

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred…

We describe how hierarchical concepts can be represented in three types of layered neural networks. The aim is to support recognition of the concepts when partial information about the concepts is presented, and also when some of the…

New cells are generated throughout life and integrate into the hippocampus via the process of adult neurogenesis. Epileptogenic brain injury induces many structural changes in the hippocampus, including the death of interneurons and altered…

Models of cortical neuronal circuits commonly depend on inhibitory feedback to control gain, provide signal normalization, and to selectively amplify signals using winner-take-all (WTA) dynamics. Such models generally assume that excitatory…

Gated networks are networks that contain gating connections, in which the outputs of at least two neurons are multiplied. Initially, gated networks were used to learn relationships between two input sources, such as pixels from two images.…

The static synaptic connectivity of neuronal circuits stands in direct contrast to the dynamics of their function. As in changing community interactions, different neurons can participate actively in various combinations to effect behaviors…

Encoder-decoder networks with attention have proven to be a powerful way to solve many sequence-to-sequence tasks. In these networks, attention aligns encoder and decoder states and is often used for visualizing network behavior. However,…

The rapid-chase theory of response priming defines a set of behavioral criteria that indicate feedforward processing of visual stimulus features rather than recurrent processing. These feedforward criteria are strong predictions from a…

Machine learning has made tremendous progress in recent years and received large amounts of public attention. Though we are still far from designing a full artificially intelligent agent, machine learning has brought us many applications in…

Many natural systems are organized as networks, in which the nodes (be they cells, individuals or populations) interact in a time-dependent fashion. The dynamic behavior of these networks depends on how these nodes are connected, which can…

As implied by the plethora of literature on graph rewiring, the choice of computational graph employed by a neural network can make a significant impact on its downstream performance. Certain effects related to the computational graph, such…

Higher order artificial neurons whose outputs are computed by applying an activation function to a higher order multinomial function of the inputs have been considered in the past, but did not gain acceptance due to the extra parameters and…

Recurrent neural networks are powerful tools for understanding and modeling computation and representation by populations of neurons. Continuous-variable or "rate" model networks have been analyzed and applied extensively for these…

Neurons in the primary visual cortex are more or less selective for the orientation of a light bar used for stimulation. A broad distribution of individual grades of orientation selectivity has in fact been reported in all species. A…

A sufficiently large information flux in recurrent neural networks, quantified by the mutual information between successive network states, is considered a prerequisite for rich information processing capabilities. This raises the question…

This work presents an analysis of the effectiveness of using standard shallow feed-forward networks to mimic the behavior of the attention mechanism in the original Transformer model, a state-of-the-art architecture for sequence-to-sequence…