Related papers: Two-argument activation functions learn soft XOR o…
In human perception and cognition, a fundamental operation that brains perform is interpretation: constructing coherent neural states from noisy, incomplete, and intrinsically ambiguous evidence. The problem of interpretation is well…
Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural…
The implementation of analog neural network and online analog learning circuits based on memristive crossbar has been intensively explored in recent years. The implementation of various activation functions is important, especially for deep…
Neural networks are composed of neurons and synapses, which are responsible for learning in a slow adaptive dynamical process. Here we experimentally show that neurons act like independent anisotropic multiplex hubs, which relay and mute…
Non-von Neumann computational hardware, based on neuron-inspired, non-linear elements connected via linear, weighted synapses -- so-called neuromorphic systems -- is a viable computational substrate. Since neuromorphic systems have been…
We consider a model of an artificial neural network that uses quantum-mechanical particles in a two-humped potential as a neuron. To simulate such a quantum-mechanical system the Monte-Carlo integration method is used. A form of the…
We give a polynomial-time algorithm for learning neural networks with one layer of sigmoids feeding into any Lipschitz, monotone activation function (e.g., sigmoid or ReLU). We make no assumptions on the structure of the network, and the…
In this article, we consider designs of simple analog artificial neural networks based on adiabatic Josephson cells with a sigmoid activation function. A new approach based on the gradient descent method is developed to adjust the circuit…
The promising performance increase offered by quantum computing has led to the idea of applying it to neural networks. Studies in this regard can be divided into two main categories: simulating quantum neural networks with the standard…
For complex life to evolve, a sophisticated nervous system for handling its complexities was fundamental. The demand resulted in the emergence of brain's computational facility, the neuronal network. This facet of the brain is attested…
Large language models (LLMs) have shown strong arithmetic reasoning capabilities when prompted with Chain-of-Thought (CoT) prompts. However, we have only a limited understanding of how they are processed by LLMs. To demystify it, prior work…
Activation functions are non-linearities in neural networks that allow them to learn complex mapping between inputs and outputs. Typical choices for activation functions are ReLU, Tanh, Sigmoid etc., where the choice generally depends on…
The backpropagation algorithm is an invaluable tool for training artificial neural networks; however, because of a weight sharing requirement, it does not provide a plausible model of brain function. Here, in the context of a two-layer…
Deep learning (DL) has big-data processing capabilities that are as good, or even better, than those of humans in many real-world domains, but at the cost of high energy requirements that may be unsustainable in some applications and of…
Recently, with the rapid development of technology, there are a lot of applications require to achieve low-cost learning. However the computational power of classical artificial neural networks, they are not capable to provide low-cost…
We propose a model for description of activity-dependent evolution and self-wiring between binary neurons. Specifically, this model can be used for investigation of growth of neuronal connectivity in the developing neocortex. By using…
Neural network acoustic models have significantly advanced state of the art speech recognition over the past few years. However, they are usually computationally expensive due to the large number of matrix-vector multiplications and…
Neurons, as eukaryotic cells, have powerful internal computation capabilities. One neuron can have many distinct states, and brains can use this capability. Processes of neuron growth and maintenance use chemical signalling between cell…
We investigate the dynamics of a network consisting of an array of identical cortical units with nearest neighbor interactions under periodic arousal. Each unit consists of two interconnected populations of neurons tuned to a state in which…
The human brain is a dynamical system whose extremely complex sensor-driven neural processes give rise to conceptual, logical cognition. Understanding the interplay between nonlinear neural dynamics and concept-level cognition remains a…