Related papers: A layered neural network with three-state neurons …

Recurrent neural network architectures can have useful computational properties, with complex temporal dynamics and input-sensitive attractor states. However, evaluation of recurrent dynamic architectures requires solution of systems of…

We study the stability and information encoding capacity of synchronized states in a neuronal network model that represents part of thalamic circuitry. Our model neurons have a Hodgkin-Huxley-type low threshold Calcium channel, display post…

Computational modeling is becoming a widely used methodology in modern neuroscience. However, as the complexity of the phenomena under study increases, the analysis of the results emerging from the simulations concomitantly becomes more…

Linear layers in neural networks (NNs) trained by gradient descent can be expressed as a key-value memory system which stores all training datapoints and the initial weights, and produces outputs using unnormalised dot attention over the…

The steady-state firing rate and firing-rate response of the leaky and exponential integrate-and-fire models receiving synaptic shot noise with excitatory and inhibitory reversal potentials is examined. For the particular case where the…

Equations of State model relations between thermodynamic variables and are ubiquitous in scientific modelling, appearing in modern day applications ranging from Astrophysics to Climate Science. The three desired properties of a general…

We develop a general theory of synaptic neural balance and how it can emerge or be enforced in neural networks. For a given regularizer, a neuron is said to be in balance if the total cost of its input weights is equal to the total cost of…

We describe a novel family of models of multi- layer feedforward neural networks in which the activation functions are encoded via penalties in the training problem. Our approach is based on representing a non-decreasing activation function…

Information measures are often used to assess the efficacy of neural networks, and learning rules can be derived through optimization procedures on such measures. In biological neural networks, computation is restricted by the amount of…

Neural networks can be compressed to reduce memory and computational requirements, or to increase accuracy by facilitating the use of a larger base architecture. In this paper we focus on pruning individual neurons, which can simultaneously…

We present a neuronal network model inspired by the Ising model, where each neuron is a binary spin ($s_i = \pm1$) interacting with its neighbors on a 2D lattice. Updates are asynchronous and follow Metropolis dynamics, with a…

Deep learning's success comes with growing energy demands, raising concerns about the long-term sustainability of the field. Spiking neural networks, inspired by biological neurons, offer a promising alternative with potential computational…

Spiking Neural Networks (SNNs) have attracted enormous research interest due to temporal information processing capability, low power consumption, and high biological plausibility. However, the formulation of efficient and high-performance…

Recent works demonstrate that early layers in a neural network contain useful information for prediction. Inspired by this, we show that extending temperature scaling across all layers improves both calibration and accuracy. We call this…

This paper considers optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that require the objectives across the network to lie in…

Learning requires the traversal of inherently distinct cognitive states to produce behavioral adaptation. Yet, tools to explicitly measure these states with non-invasive imaging -- and to assess their dynamics during learning -- remain…

In this paper, we propose a shot noise-based leaky integrated and firing neuron model and provide a detailed analysis of the performance of this model compared to the traditional diffusion approximated model. In theoretical neuroscience,…

Rapid development in numerical modelling of materials and the complexity of new models increases quickly together with their computational demands. Despite the growing performance of modern computers and clusters, calibration of such models…

We extensively explore networks of weakly unbalanced, leaky integrate-and-fire (LIF) neurons for different coupling strength, connectivity, and by varying the degree of refractoriness, as well as the delay in the spike transmission. We find…

Applications that generate huge amounts of data in the form of fast streams are becoming increasingly prevalent, being therefore necessary to learn in an online manner. These conditions usually impose memory and processing time…