Related papers: Perceptrons from Memristors
The increasing computational demands of deep learning models pose significant challenges for edge devices. To address this, we propose a memristor-based circuit design for MobileNetV3, specifically for image classification tasks. Our design…
The transformer architecture and variants presented remarkable success across many machine learning tasks in recent years. This success is intrinsically related to the capability of handling long sequences and the presence of…
Recently, we proposed to transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average, and use separate shortcut connections to model the linear dependencies instead. We…
Inner products of neural network feature maps arise in a wide variety of machine learning frameworks as a method of modeling relations between inputs. This work studies the approximation properties of inner products of neural networks. It…
Large language models often expose their brittleness in reasoning tasks, especially while executing long chains of reasoning over context. We propose MemReasoner, a new and simple memory-augmented LLM architecture, in which the memory…
Perceptrons are the basic computational unit of artificial neural networks, as they model the activation mechanism of an output neuron due to incoming signals from its neighbours. As linear classifiers, they play an important role in the…
General purpose computing systems are used for a large variety of applications. Extensive supports for flexibility in these systems limit their energy efficiencies. Neural networks, including deep networks, are widely used for signal…
Edge devices operating in dynamic environments critically need the ability to continually learn without catastrophic forgetting. The strict resource constraints in these devices pose a major challenge to achieve this, as continual learning…
The concept of a recently proposed Forward-Forward learning algorithm for fully connected artificial neural networks is applied to a single multi output perceptron for classification. The parameters of the system are trained with respect to…
Mapping neuro-inspired algorithms to sensor backplanes of on-chip hardware require shifting the signal processing from digital to the analog domain, demanding memory technologies beyond conventional CMOS binary storage units. Using…
We suggest electronic circuits with memristors (resistors with memory) that operate as memcapacitors (capacitors with memory) and meminductors (inductors with memory). Using a memristor emulator, the suggested circuits have been built and…
Feed-forward, fully-connected Artificial Neural Networks (ANNs) or the so-called Multi-Layer Perceptrons (MLPs) are well-known universal approximators. However, their learning performance varies significantly depending on the function or…
Mean-field models provide a link between microscopic neuronal activity and macroscopic brain dynamics. Their derivation depends on simplifying assumptions, such as all-to-all connectivity, limiting their biological realism. To overcome…
Conceptual memristors have recently gathered wider interest due to their diverse application in non-von Neumann computing, machine learning, neuromorphic computing, and chaotic circuits. We introduce a compact CMOS circuit that emulates…
The thesis investigates the utilization of memristive and memcapacitive crossbar arrays in low-power machine learning accelerators, offering a comprehensive co-design framework for deep neural networks (DNN). The model, implemented through…
Neuromorphic circuits mimic partial functionalities of brain in a bio-inspired information processing sense in order to achieve similar efficiencies as biological systems. While there are common mathematical models for neurons, which can be…
Due to many limitations of Von Neumann architecture such as speed, memory bandwidth, efficiency of global interconnects and increase in the application of artificial neural network, researchers have been pushed to look into alternative…
The universal approximation theorem states that a neural network with one hidden layer can approximate continuous functions on compact sets with any desired precision. This theorem supports using neural networks for various applications,…
Redox-based nanoionic resistive memory cells (ReRAMs) are one of the most promising emerging nano-devices for future information technology with applications for memory, logic and neuromorphic computing. Recently, the serendipitous…
The implementation of analog neural network and online analog learning circuits based on memristive crossbar has been intensively explored in recent years. The implementation of various activation functions is important, especially for deep…