Related papers: Perceptron Linear Function Design with CMOS-Memris…

In the current research of neural networks, the activation function is manually specified by human and not able to change themselves during training. This paper focus on how to make the activation function trainable for deep neural…

Memory effects are ubiquitous in nature and the class of memory circuit elements - which includes memristors, memcapacitors and meminductors - shows great potential to understand and simulate the associated fundamental physical processes.…

The widespread application of artificial neural networks has prompted researchers to experiment with FPGA and customized ASIC designs to speed up their computation. These implementation efforts have generally focused on weight…

Attention improves representation learning over RNNs, but its discrete nature limits continuous-time (CT) modeling. We introduce Neuronal Attention Circuit (NAC), a novel, biologically inspired CT-Attention mechanism that reformulates…

We design and model a single-layer, passive, all-optical silicon photonics neural network to mitigate optical link nonlinearities. The network nodes are formed by silicon microring resonators whose transfer function has been experimentally…

Once referred to as the missing circuit component, memristor has come long way across to be recognized and taken as important to future circuit designs. The memristor due to its ability to memorize the state, switch between different…

Traditional analytical reflectance models, while compact and interpretable, lack the capacity to accurately represent physical measurements. Recent neural models, which closely fit input data, are less generalizable and often more expensive…

Traditional Convolutional Neural Networks (CNNs) typically use the same activation function (usually ReLU) for all neurons with non-linear mapping operations. For example, the deep convolutional architecture Inception-v4 uses ReLU. To…

The binary perceptron is the simplest artificial neural network formed by $N$ input units and one output unit, with the neural states and the synaptic weights all restricted to $\pm 1$ values. The task in the teacher--student scenario is to…

Optical artificial neural networks (ONNs), analog computing hardware tailored for machine learning, have significant potential for ultra-high computing speed and energy efficiency. We propose a new approach to architectures for ONNs based…

Spiking neural networks (SNNs) are being explored in an attempt to mimic brain's capability to learn and recognize at low power. Crossbar architecture with highly scalable Resistive RAM or RRAM array serving as synaptic weights and neuronal…

Analog computing based on memristor technology is a promising solution to accelerating the inference phase of deep neural networks (DNNs). A fundamental problem is to map an arbitrary matrix to a memristor crossbar array (MCA) while…

We demonstrate the first hardware implementation of an oscillatory neural network (ONN) utilizing resistive memory (ReRAM) for coupling elements. A ReRAM crossbar array chip, integrated into the Back End of Line (BEOL) of CMOS technology,…

The superior density of passive analog-grade memristive crossbars may enable storing large synaptic weight matrices directly on specialized neuromorphic chips, thus avoiding costly off-chip communication. To ensure efficient use of such…

The effectiveness of deep neural architectures has been widely supported in terms of both experimental and foundational principles. There is also clear evidence that the activation function (e.g. the rectifier and the LSTM units) plays a…

A resistive memory network that has no crossover wiring is proposed to overcome the hardware limitations to size and functional complexity that is associated with conventional analogue neural networks. The proposed memory network is based…

We extend the hyperplane arrangement framework for neural network expressivity from the braid to discriminantal arrangements. Compatible piecewise linear functions are characterized by circuit relations and admit a matroidal description via…

This paper presents a highly speculative model encompassing the cortex, thalamus, and hippocampus of the mammalian brain. While the majority of computational neuroscience models are founded upon empirical evidence, this model is predicated…

We introduce an electro-optic hardware platform for nonlinear activation functions in optical neural networks. The optical-to-optical nonlinearity operates by converting a small portion of the input optical signal into an analog electric…

A pivotal aspect in the design of neural networks lies in selecting activation functions, crucial for introducing nonlinear structures that capture intricate input-output patterns. While the effectiveness of adaptive or trainable activation…