Related papers: Adiabatic Superconducting Artificial Neural Networ…

Artificial Neural Networks have emerged as an important tool for classification and have been widely used to classify a non-linear separable pattern. The most popular artificial neural networks model is a Multilayer Perceptron (MLP) as it…

The ability of neural networks to represent more features than neurons makes interpreting them challenging. This phenomenon, known as superposition, has spurred efforts to find architectures that are more interpretable than standard…

Modern deep learning architectures are ordinarily performed on high-performance computing facilities due to the large size of the input features and complexity of its model. This paper proposes traditional multilayer perceptrons (MLP) with…

We show that MLP layers in transformer language models perform binary routing of continuous signals: the decision of whether a token needs nonlinear processing is well-captured by binary neuron activations, even though the signals being…

A multilayer perceptron (MLP) is typically made of multiple fully connected layers with nonlinear activation functions. There have been several approaches to make them better (e.g. faster convergence, better convergence limit, etc.). But…

Unlike humans, who are capable of continual learning over their lifetimes, artificial neural networks have long been known to suffer from a phenomenon known as catastrophic forgetting, whereby new learning can lead to abrupt erasure of…

Neurons in real brains are enormously complex computational units. Among other things, they're responsible for transforming inbound electro-chemical vectors into outbound action potentials, updating the strengths of intermediate synapses,…

Embedding based models have been the state of the art in collaborative filtering for over a decade. Traditionally, the dot product or higher order equivalents have been used to combine two or more embeddings, e.g., most notably in matrix…

Replacing electrons with photons is a compelling route towards light-speed, highly parallel, and low-power artificial intelligence computing. Recently, all-optical diffractive neural deep neural networks have been demonstrated. However, the…

Superconducting computing promises enhanced computational power in both classical and quantum approaches. Yet, scalable and fast superconducting memories are not implemented. Here, we propose a fully superconducting memory cell based on the…

The highly parallel process in the neuron networks is mediated through a mass of synaptic interconnections. Mimicking single synapse behaviors and highly paralleled neural networks has become more and more fascinating and important. Here,…

In this paper, we described and developed a framework for Multilayer Perceptron (MLP) to work on low level image processing, where MLP will be used to perform image super-resolution. Meanwhile, MLP are trained with different types of images…

Graph Neural Networks (GNNs) have become a dominant approach to learning graph representations, primarily because of their message-passing mechanisms. However, GNNs typically adopt a fixed aggregator function such as Mean, Max, or Sum…

Due to the massive parallel computing capability and outstanding image and signal processing performance, cellular neural network (CNN) is one promising type of non-Boolean computing system that can outperform the traditional digital logic…

The definition of a Neural Network architecture is one of the most critical and challenging tasks to perform. In this paper, we propose ParallelMLPs. ParallelMLPs is a procedure to enable the training of several independent Multilayer…

Memristors have been widely studied as artificial synapses in neuromorphic circuits, due to their functional similarity with biological synapses, low operating power, and high integration density. In this work, a memristive synapse,…

Multimode optical fibres are hair-thin strands of glass that efficiently transport light. They promise next-generation medical endoscopes that provide unprecedented sub-cellular image resolution deep inside the body. However, confining…

Dendrites are crucial structures for computation of an individual neuron. It has been shown that the dynamics of a biological neuron with dendrites can be approximated by artificial neural networks (ANN) with deep structure. However, it…

Conventional digital computation is rapidly approaching physical limits for speed and energy dissipation. Here we fabricate and test a simple neuromorphic circuit that models neuronal somas, axons and synapses with superconducting Josephson…

We design and model a single-layer, passive, all-optical silicon photonics neural network to mitigate optical link nonlinearities. The network nodes are formed by silicon microring resonators whose transfer function has been experimentally…