相关论文: Critical Echo State Networks that Anticipate Input…

The goal of this paper is to investigate the theoretical properties, the training algorithm, and the predictive control applications of Echo State Networks (ESNs), a particular kind of Recurrent Neural Networks. First, a condition…

Neural machine translation (NMT) models learn representations containing substantial linguistic information. However, it is not clear if such information is fully distributed or if some of it can be attributed to individual neurons. We…

A model of sensory information processing is presented. The model assumes that learning of internal (hidden) generative models, which can predict the future and evaluate the precision of that prediction, is of central importance for…

We present neural machine translation (NMT) models inspired by echo state network (ESN), named Echo State NMT (ESNMT), in which the encoder and decoder layer weights are randomly generated then fixed throughout training. We show that even…

Dynamic temporal graphs represent evolving relations between entities, e.g. interactions between social network users or infection spreading. We propose an extension of graph echo state networks for the efficient processing of dynamic…

The problem of optimising a network of discretely firing neurons is addressed. An objective function is introduced which measures the average number of bits that are needed for the network to encode its state. When this is minimised, it is…

Cells are known to utilize biochemical noise to probabilistically switch between distinct gene expression states. We demonstrate that such noise-driven switching is dominated by tails of probability distributions and is therefore…

Differentiable neural computers extend artificial neural networks with an explicit memory without interference, thus enabling the model to perform classic computation tasks such as graph traversal. However, such models are difficult to…

Among the various architectures of Recurrent Neural Networks, Echo State Networks (ESNs) emerged due to their simplified and inexpensive training procedure. These networks are known to be sensitive to the setting of hyper-parameters, which…

In this paper we introduce a new framework to train an Echo State Network to predict real valued time-series. The method consists in projecting the output of the internal layer of the network on a space with lower dimensionality, before…

A key question in brain sciences is how to identify time-evolving functional connectivity, such as that obtained from recordings of neuronal activity over time. We wish to explain the observed phenomena in terms of latent states which, in…

The dynamics of competing opinions in social network play an important role in society, with many applications in diverse social contexts as consensus, elections, morality and so on. Here we study a model of interacting agents connected in…

Neurons, as eukaryotic cells, have powerful internal computation capabilities. One neuron can have many distinct states, and brains can use this capability. Processes of neuron growth and maintenance use chemical signalling between cell…

Predictive coding (PC) is an influential theory in computational neuroscience, which argues that the cortex forms unsupervised world models by implementing a hierarchical process of prediction error minimization. PC networks (PCNs) are…

Many physical and biological systems can be studied using complex network theory, a new statistical physics understanding of graph theory. The recent application of complex network theory to the study of functional brain networks generated…

Living cells can enhance their fitness by anticipating environmental change. We study how accurately linear signaling networks in cells can predict future signals. We find that maximal predictive power results from a combination of…

Supralinear and sublinear pre-synaptic and dendritic integration is considered to be responsible for nonlinear computation power of biological neurons, emphasizing the role of nonlinear integration as opposed to nonlinear output…

This paper proposes a novel and interpretable recurrent neural-network structure using the echo-state network (ESN) paradigm for time-series prediction. While the traditional ESNs perform well for dynamical systems prediction, it needs a…

The critical brain hypothesis states that there are information processing advantages for neuronal networks working close to the critical region of a phase transition. If this is true, we must ask how the networks achieve and maintain this…

The presence of both critical behavior and oscillating patterns in brain dynamics is a very interesting issue. In this paper, we consider a model for a neuron population, where each neuron is modeled by an over-damped rotator. We find that…